Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

RDBMS to Graphs

1 191 vues

Publié le

Relational databases were conceived to digitize paper forms and automate well-structured business processes, and still have their uses. But RDBMS cannot model or store data and its relationships without complexity, which means performance degrades with the increasing number and levels of data relationships and data size. Additionally, new types of data and data relationships require schema redesign that increases time to market.

A graph database like Neo4j naturally stores, manages, analyzes, and uses data within the context of connections meaning Neo4j provides faster query performance and vastly improved flexibility in handling complex hierarchies than SQL. Join this webinar to learn why companies are shifting away from RDBMS towards graphs to unlock the business value in their data relationships

Publié dans : Technologie
  • Login to see the comments

RDBMS to Graphs

  1. 1. (RDBMS) -[:TO]->(Graph) An Introduction To Graphs and Neo4j William Lyon @lyonwj
  2. 2. William Lyon Developer Relations Engineer @neo4j will@neo4j.com @lyonwj lyonwj.com
  3. 3. AGENDA • Use Cases • SQL Pains • Building a Neo4j Application • Moving from RDBMS -> Graph Models • Walk through an Example • Creating Data in Graphs • Querying Data
  4. 4. The Traditional Approach Towards Data
  5. 5. SYSTEMS OF RECORD
  6. 6. RDBMS SYSTEMS OF RECORD
  7. 7. HR-tools Supply Payments Logistics CRM Support TRADITIONAL DATA STRUCTURE RDBMS RDBMS RDBMSRDBMS RDBMS RDBMS
  8. 8. SHIFT TOWARDS SYSTEMS OF ENGAGEMENT
  9. 9. Users Engaging With DevicesUsers Engaging With Users Devices Engaging With Devices SYSTEMS OF ENGAGEMENT
  10. 10. SHIFT TOWARDS SYSTEMS OF ENGAGEMENT You are here! Data volume
  11. 11. SYSTEMS OF RECORD Relational Database Model Structured Pre-computed Based on rigid rules SYSTEMS OF ENGAGEMENT NoSQL Database Model Highly Flexible Real-Time Queries Highly Contextual
  12. 12. SYSTEMS OF RECORD
  13. 13. SYSTEMS OF ENGAGEMENT This is data modeled as a graph!
  14. 14. WHAT IS A GRAPH?
  15. 15. A Graph Is NODE NODE NODE RELATIONSHIP RELATIONSHIP RELATIONSHIP
  16. 16. A Graph Is Connected Data
  17. 17. ROAD TRAFFIC LIGHTS
  18. 18. HAS AVAILABLE HOTEL ROOMS AVAILABLE
  19. 19. CREATING COMPETITIVE ADVANTAGE FROM THE USE OF DATA
  20. 20. ACCOUNT HOLDER 2 ACCOUNT HOLDER 1 ACCOUNT HOLDER 3 CREDIT CARD BANK ACCOUNT BANK ACCOUNT BANK ACCOUNT ADDRESS PHONE NUMBER PHONE NUMBER SSN 2 UNSECURE LOAN SSN 2 UNSECURE LOAN CREDIT CARD
  21. 21. Intuitivness Speed Agility
  22. 22. Intuitiveness Speed Agility
  23. 23. Intuitiveness
  24. 24. Intuitivness Speed Agility
  25. 25. Speed “We found Neo4j to be literally thousands of times faster than our prior MySQL solution, with queries that require 10-100 times less code. Today, Neo4j provides eBay with functionality that was previously impossible.” - Volker Pacher, Senior Developer “Minutes to milliseconds” performance Queries up to 1000x faster than RDBMS or other NoSQL
  26. 26. Intuitivness Speed Agility
  27. 27. A Naturally Adaptive Model A Query Language Designed for Connectedness + =Agility
  28. 28. Cypher Typical Complex SQL Join The Same Query using Cypher MATCH (boss)-[:MANAGES*0..3]->(sub), (sub)-[:MANAGES*1..3]->(report) WHERE boss.name = “John Doe” RETURN sub.name AS Subordinate, 
 count(report) AS Total Project Impact Less time writing queries Less time debugging queries Code that’s easier to read
  29. 29. NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations
  30. 30. NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations GRAPH THINKING: Real Time Recommendations VIEWED VIEWED BOUGHT VIEWED BOUGHT BOUGHT BOUGHT BOUGHT
  31. 31. “As the current market leader in graph databases, and with enterprise features for scalability and availability, Neo4j is the right choice to meet our demands.” Marcos Wada Software Developer, Walmart NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations
  32. 32. NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations GRAPH THINKING: Master Data Management MANAGES MANAGES LEADS REGION M ANAG ES MANAGES REGION LEADS LEADS COLLABORATES
  33. 33. Neo4j is the heart of Cisco HMP: used for governance and single source of truth and a one-stop shop for all of Cisco’s hierarchies. NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations
  34. 34. NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations GRAPH THINKING: Master Data Management Solu%on Support Case Support Case Knowledge Base Ar%cle Message Knowledge Base Ar%cle Knowledge Base Ar%cle Neo4j is the heart of Cisco’s Helpdesk Solution too.
  35. 35. NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations GRAPH THINKING: Fraud Detection O PENED_ACCO UNT HAS IS_ISSUED HAS LIVES LIVES IS_ISSUED OPENED_ACCOUNT
  36. 36. “Graph databases offer new methods of uncovering fraud rings and other sophisticated scams with a high-level of accuracy, and are capable of stopping advanced fraud scenarios in real-time.” Gorka Sadowski Cyber Security Expert NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations
  37. 37. GRAPH THINKING: Graph Based Search NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations PUBLISH INCLUDE INCLUDE CREATE CAPTURE IN IN SOURCE USES USES IN IN USES SOURCE SOURCE
  38. 38. Uses Neo4j to manage the digital assets inside of its next generation in-flight entertainment system. NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations
  39. 39. NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations BROWSES CONNECTS BRIDGES ROUTES POWERS ROUTES POWERS POWERS HOSTS QUERIES GRAPH THINKING: Network & IT-Operations
  40. 40. Uses Neo4j for network topology analysis for big telco service providers NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations
  41. 41. GRAPH THINKING: Identity And Access Management NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations TRUSTS TRUSTS ID ID AUTHENTICATES AUTHENTICATES O W NS OWNS CAN_READ
  42. 42. UBS was the recipient of the 2014 Graphie Award for “Best Identify And Access Management App” NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations
  43. 43. Neo4j Adoption by Selected Verticals SOFTWARE FINANCIAL SERVICES RETAIL MEDIA & BROADCASTING SOCIAL NETWORKS TELECOM HEALTHCARE
  44. 44. SQL Day in the Life of a RDBMS Developer
  45. 45. SELECT p.name, c.country, c.leader, p.hair, u.name, u.pres, u.state FROM people p LEFT JOIN country c ON c.ID=p.country LEFT JOIN uni u ON p.uni=u.id WHERE u.state=‘CT’
  46. 46. JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN
  47. 47. JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN
  48. 48. JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN JOIN
  49. 49. • Complex to model and store relationships • Performance degrades with increases in data • Queries get long and complex • Maintenance is painful SQL Pains
  50. 50. • Easy to model and store relationships • Performance of relationship traversal remains constant with growth in data size • Queries are shortened and more readable • Adding additional properties and relationships can be done on the fly - no migrations Graph Gains
  51. 51. Relational Versus Graph Models Relational Model Graph Model KNOWS KNOWS KNOWS ANDREAS TOBIAS MICA DELIA Person FriendPerson-Friend ANDREAS DELIA TOBIAS MICA
  52. 52. What does this Graph look like?
  53. 53. CYPHER Ann DanLoves
  54. 54. Property Graph Model CREATE (:Person { name:“Dan”} ) - [:LOVES]-> (:Person { name:“Ann”} ) LOVES LABEL PROPERTY NODE NODE LABEL PROPERTY
  55. 55. MATCH (p:Person)-[:WENT_TO]->(u:Uni), (p)-[:LIVES_IN]->(c:Country), (u)-[:LED_BY]->(l:Leader), (u)-[:LOCATED_IN]->(s:State) WHERE s.abbr = ‘CT’ RETURN p.name, c.country, c.leader, p.hair, u.name, l.name, s.abbr
  56. 56. How do you use Neo4j? CREATE MODEL + LOAD DATA QUERY DATA
  57. 57. How do you use Neo4j?
  58. 58. How do you use Neo4j?
  59. 59. Language Drivers
  60. 60. Language Drivers
  61. 61. User Defined Procedures
  62. 62. Architectural Options Data Storage and Business Rules Execu5on Data Mining and Aggrega5on Applica'on Graph Database Cluster Neo4j Neo4j Neo4j Ad Hoc Analysis Bulk Analy'c Infrastructure Hadoop, EDW … Data Scien'st End User Databases Rela5onal NoSQL Hadoop
  63. 63. RDBMS to Graph Options MIGRATE ALL DATA MIGRATE SUBSET DUPLICATE SUBSET Non-Graph Queries Graph Queries Graph Queries Non-Graph Queries All Queries Rela3onal Database Graph Database Application Application Application Non Graph Data All Data
  64. 64. FROM RDBMS TO GRAPHS
  65. 65. Northwind
  66. 66. Northwind - the canonical RDBMS Example
  67. 67. ( )-[:TO]->(Graph)
  68. 68. ( )-[:IS_BETTER_AS]->(Graph)
  69. 69. Starting with the ER Diagram
  70. 70. Locate the Foreign Keys
  71. 71. Drop the Foreign Keys
  72. 72. Find the JOIN Tables
  73. 73. (Simple) JOIN Tables Become Relationships
  74. 74. Attributed JOIN Tables -> Relationships with Properties
  75. 75. Querying a Subset Today
  76. 76. As a Graph
  77. 77. QUERYING THE GRAPH
  78. 78. using openCypher
  79. 79. Property Graph Model CREATE (:Employee{ firstName:“Steven”} ) -[:REPORTS_TO]-> (:Employee{ firstName:“Andrew”} ) REPORTS_TO Steven Andrew LABEL PROPERTY NODE NODE LABEL PROPERTY
  80. 80. Who do people report to? MATCH (e:Employee)<-[:REPORTS_TO]-(sub:Employee) RETURN *
  81. 81. Who do people report to?
  82. 82. Who do people report to? MATCH (e:Employee)<-[:REPORTS_TO]-(sub:Employee) RETURN e.employeeID AS managerID, e.firstName AS managerName, sub.employeeID AS employeeID, sub.firstName AS employeeName;
  83. 83. Who do people report to?
  84. 84. Who does Robert report to? MATCH p=(e:Employee)<-[:REPORTS_TO]-(sub:Employee) WHERE sub.firstName = ‘Robert’ RETURN p
  85. 85. Who does Robert report to?
  86. 86. What is Robert’s reporting chain? MATCH p=(e:Employee)<-[:REPORTS_TO*]-(sub:Employee) WHERE sub.firstName = ‘Robert’ RETURN p
  87. 87. What is Robert’s reporting chain?
  88. 88. Who’s the Big Boss? MATCH (e:Employee) WHERE NOT (e)-[:REPORTS_TO]->() RETURN e.firstName as bigBoss
  89. 89. Who’s the Big Boss?
  90. 90. Product Cross-Selling MATCH (choc:Product {productName: 'Chocolade'}) <-[:INCLUDES]-(:Order)<-[:SOLD]-(employee), (employee)-[:SOLD]->(o2)-[:INCLUDES]->(other:Product) RETURN employee.firstName, other.productName, COUNT(DISTINCT o2) as count ORDER BY count DESC LIMIT 5;
  91. 91. Product Cross-Selling
  92. 92. https://neo4j.com/graphacademy/online-training/
  93. 93. POWERING AN APP
  94. 94. http://network.graphdemos.com/
  95. 95. Simple App
  96. 96. Simple Python Code
  97. 97. Simple Python Code
  98. 98. Simple Python Code
  99. 99. Simple Python Code
  100. 100. LOADING OUR DATA
  101. 101. CSV
  102. 102. CSV files for Northwind
  103. 103. CSV files for Northwind
  104. 104. 3 Steps to Creating the Graph IMPORT NODES CREATE INDEXES IMPORT RELATIONSHIPS
  105. 105. Importing Nodes // Create customers USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM "https:// raw.githubusercontent.com/neo4j-contrib/developer-resources/ gh-pages/data/northwind/customers.csv" AS row CREATE (:Customer {companyName: row.CompanyName, customerID: row.CustomerID, fax: row.Fax, phone: row.Phone}); // Create products USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM "https:// raw.githubusercontent.com/neo4j-contrib/developer-resources/ gh-pages/data/northwind/products.csv" AS row CREATE (:Product {productName: row.ProductName, productID: row.ProductID, unitPrice: toFloat(row.UnitPrice)});
  106. 106. Importing Nodes // Create suppliers USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM "https:// raw.githubusercontent.com/neo4j-contrib/developer-resources/ gh-pages/data/northwind/suppliers.csv" AS row CREATE (:Supplier {companyName: row.CompanyName, supplierID: row.SupplierID}); // Create employees USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM "https:// raw.githubusercontent.com/neo4j-contrib/developer-resources/ gh-pages/data/northwind/employees.csv" AS row CREATE (:Employee {employeeID:row.EmployeeID, firstName: row.FirstName, lastName: row.LastName, title: row.Title});
  107. 107. Importing Nodes // Create categories USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM "https:// raw.githubusercontent.com/neo4j-contrib/developer-resources/ gh-pages/data/northwind/categories.csv" AS row CREATE (:Category {categoryID: row.CategoryID, categoryName: row.CategoryName, description: row.Description}); // Create orders USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM "https:// raw.githubusercontent.com/neo4j-contrib/developer-resources/ gh-pages/data/northwind/orders.csv" AS row MERGE (order:Order {orderID: row.OrderID}) ON CREATE SET order.shipName = row.ShipName;
  108. 108. Creating Indexes CREATE INDEX ON :Product(productID); CREATE INDEX ON :Product(productName); CREATE INDEX ON :Category(categoryID); CREATE INDEX ON :Employee(employeeID); CREATE INDEX ON :Supplier(supplierID); CREATE INDEX ON :Customer(customerID); CREATE INDEX ON :Customer(customerName);
  109. 109. Creating Relationships USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM "https:// raw.githubusercontent.com/neo4j-contrib/developer-resources/ gh-pages/data/northwind/orders.csv" AS row MATCH (order:Order {orderID: row.OrderID}) MATCH (customer:Customer {customerID: row.CustomerID}) MERGE (customer)-[:PURCHASED]->(order); USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM "https:// raw.githubusercontent.com/neo4j-contrib/developer-resources/ gh-pages/data/northwind/products.csv" AS row MATCH (product:Product {productID: row.ProductID}) MATCH (supplier:Supplier {supplierID: row.SupplierID}) MERGE (supplier)-[:SUPPLIES]->(product);
  110. 110. Creating Relationships USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM "https://raw.githubusercontent.com/neo4j- contrib/developer-resources/gh-pages/data/northwind/orders.csv" AS row MATCH (order:Order {orderID: row.OrderID}) MATCH (product:Product {productID: row.ProductID}) MERGE (order)-[pu:INCLUDES]->(product) ON CREATE SET pu.unitPrice = toFloat(row.UnitPrice), pu.quantity = toFloat(row.Quantity); USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM "https://raw.githubusercontent.com/neo4j- contrib/developer-resources/gh-pages/data/northwind/orders.csv" AS row MATCH (order:Order {orderID: row.OrderID}) MATCH (employee:Employee {employeeID: row.EmployeeID}) MERGE (employee)-[:SOLD]->(order);
  111. 111. Creating Relationships USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM "https://raw.githubusercontent.com/ neo4j-contrib/developer-resources/gh-pages/data/northwind/ products.csv" AS row MATCH (product:Product {productID: row.ProductID}) MATCH (category:Category {categoryID: row.CategoryID}) MERGE (product)-[:PART_OF]->(category); USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM "https://raw.githubusercontent.com/ neo4j-contrib/developer-resources/gh-pages/data/northwind/ employees.csv" AS row MATCH (employee:Employee {employeeID: row.EmployeeID}) MATCH (manager:Employee {employeeID: row.ReportsTo}) MERGE (employee)-[:REPORTS_TO]->(manager);
  112. 112. High Performance LOADing neo4j-import 4.58 million things and their relationships… Loads in 100 seconds!
  113. 113. JDBC
  114. 114. JDBC apoc.load.jdbc
  115. 115. https://github.com/neo4j-contrib/neo4j-apoc-procedures
  116. 116. User Defined Procedures User-defined procedures are written in Java, deployed into the database, and called from Cypher. http://neo4j.com/docs/developer-manual/current/#procedures
  117. 117. Built-in Procedures
  118. 118. Built-in Procedures
  119. 119. User-defined Procedures https://github.com/neo4j-examples/neo4j-procedure-template
  120. 120. Apoc Procedures https://github.com/neo4j-contrib/neo4j-apoc-procedures
  121. 121. THERE’S A PROCEDURE FOR THAT https://github.com/neo4j-contrib/neo4j-apoc-procedures
  122. 122. https://neo4j-contrib.github.io/neo4j-apoc-procedures/ #_loading_data_from_web_apis_json_xml_csv
  123. 123. https://github.com/neo4j-contrib/neo4j-apoc-procedures/blob/master/readme.adoc
  124. 124. Northwind
  125. 125. Northwind
  126. 126. Graph Based Recommendations
  127. 127. Graph Based Recommendations
  128. 128. JDBC Neo4j-JDBC driver
  129. 129. ETL
  130. 130. https://neo4j.com/blog/icij-neo4j-unravel-panama-papers/
  131. 131. https://neo4j.com/blog/official-neo4j-jdbc-driver-3-0/
  132. 132. WRAPPING UP
  133. 133. Resources http://neo4j.com/developer/ Neo4j Developer Resources
  134. 134. http://graphdatabases.com/
  135. 135. http://neo4j.com/download
  136. 136. :play northwind graph
  137. 137. THANK YOU!

×