Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Polyglot Persistence with MongoDB and Neo4j

798 vues

Publié le

Learn how to enhance your application by using Neo4j and MongoDB together. Polyglot persistence is the concept of taking advantage of the strengths of different database technologies to improve functionality and enhance your application. In this webinar we will examine some use cases where it makes sense to use a document database (MongoDB) with a graph database (Neo4j) in a single application. Specifically, we will show how MongoDB can be used to provide search and browsing functionality for a product catalog while using Neo4j to provide personalized product recommendations. Finally we will look at the Neo4j Doc Manager project which facilitates syncing data from MongoDB to Neo4j to make polyglot persistence with MongoDB and Neo4j much easier.

Publié dans : Technologie
  • Soyez le premier à commenter

Polyglot Persistence with MongoDB and Neo4j

  1. 1. NoSQL Polyglot Persistence Using Neo4j and MongoDB together William Lyon @lyonwj
  2. 2. William Lyon Developer Relations Engineer @neo4j will@neo4j.com @lyonwj lyonwj.com
  3. 3. Does Neo4j work with ______ ? Languages, frameworks, databases…
  4. 4. Does Neo4j work with ______ ? Languages
  5. 5. Does Neo4j work with ______ ? Frameworks
  6. 6. Does Neo4j work with ______ ? Frameworks
  7. 7. Does Neo4j work with ______ ? Deployment
  8. 8. Does Neo4j work with ______ ? Databases
  9. 9. Does Neo4j work with ______ ? Languages, frameworks, databases…
  10. 10. Does Neo4j work with ?
  11. 11. Why? +
  12. 12. MongoDB Features • NoSQL database • Document datamodel • JSON-like documents (BSON) • Flexible data model • Horizontal scalability (sharding) • Complex queries { "session": { "title": "12 Years of Spring: An Open Source Journey", "abstract": "Spring emerged as a core open source project in early 2003 and evolved to a broad portfolio of open source projects up until 2015." }, "topics": [ "keynote", "spring" ], "room": "Auditorium", "timeslot": "Wed 29th, 09:30-10:30", "speaker": { "name": "Juergen Hoeller", "bio": "Juergen Hoeller is co-founder of the Spring Framework open source project.", "twitter": "https://twitter.com/springjuergen", "picture": "http://www.springio.net/wp-content/uploads/2014/11/ juergen_hoeller-220x220.jpeg" } }
  13. 13. MongoDB Use Cases • Product catalog • User profiles • Metadata • Content • Events • Analytics { "session": { "title": "12 Years of Spring: An Open Source Journey", "abstract": "Spring emerged as a core open source project in early 2003 and evolved to a broad portfolio of open source projects up until 2015." }, "topics": [ "keynote", "spring" ], "room": "Auditorium", "timeslot": "Wed 29th, 09:30-10:30", "speaker": { "name": "Juergen Hoeller", "bio": "Juergen Hoeller is co-founder of the Spring Framework open source project.", "twitter": "https://twitter.com/springjuergen", "picture": "http://www.springio.net/wp-content/uploads/2014/11/ juergen_hoeller-220x220.jpeg" } } https://www.mongodb.com/use-cases/
  14. 14. MongoDB Use Cases • Product catalog • User profiles • Metadata • Content • Events • Analytics { "session": { "title": "12 Years of Spring: An Open Source Journey", "abstract": "Spring emerged as a core open source project in early 2003 and evolved to a broad portfolio of open source projects up until 2015." }, "topics": [ "keynote", "spring" ], "room": "Auditorium", "timeslot": "Wed 29th, 09:30-10:30", "speaker": { "name": "Juergen Hoeller", "bio": "Juergen Hoeller is co-founder of the Spring Framework open source project.", "twitter": "https://twitter.com/springjuergen", "picture": "http://www.springio.net/wp-content/uploads/2014/11/ juergen_hoeller-220x220.jpeg" } } https://www.mongodb.com/use-cases/
  15. 15. Neo4j Graph Database • Property graph data model • Nodes and relationships • Native graph processing • Cypher query language
  16. 16. Property Graph Model Components Nodes • The objects in the graph • Can have name-value properties • Can be labeled Relationships • Relate nodes by type and direction • Can have name-value properties CAR DRIVES name: “Dan” born: May 29, 1970 twitter: “@dan” name: “Ann” born: Dec 5, 1975 since: 
 Jan 10, 2011 brand: “Volvo” model: “V70” LOVES LOVES LIVES WITH OW NS PERSON PERSON
  17. 17. Cypher: SQL for graphs CREATE (:Person { name:“Dan”} ) -[:LOVES]-> (:Person { name:“Ann”} ) LOVES Dan Ann LABEL PROPERTY NODE NODE LABEL PROPERTY
  18. 18. NEO4J USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations
  19. 19. NEO4J USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations GRAPH THINKING: Real Time Recommendations VIEWED VIEWED BOUGHT VIEWED BOUGHT BOUGHT BOUGHT BOUGHT
  20. 20. NEO4J USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations GRAPH THINKING: Master Data Management MANAGES MANAGES LEADS REGION M ANAG ES MANAGES REGION LEADS LEADS COLLABORATES
  21. 21. NEO4J USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations GRAPH THINKING: Fraud Detection O PENED_ACCO UNT HAS IS_ISSUED HAS LIVES LIVES IS_ISSUED OPENED_ACCOUNT
  22. 22. GRAPH THINKING: Graph Based Search NEO4J USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations PUBLISH INCLUDE INCLUDE CREATE CAPTURE IN IN SOURCE USES USES IN IN USES SOURCE SOURCE
  23. 23. NEO4J USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations BROWSES CONNECTS BRIDGES ROUTES POWERS ROUTES POWERS POWERS HOSTS QUERIES GRAPH THINKING: Network & IT-Operations
  24. 24. GRAPH THINKING: Identity And Access Management NEO4J USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations TRUSTS TRUSTS ID ID AUTHENTICATES AUTHENTICATES O W NS OWNS CAN_READ
  25. 25. Why? +
  26. 26. The Traditional Approach Towards Data
  27. 27. SYSTEMS OF RECORD
  28. 28. RDBMS SYSTEMS OF RECORD
  29. 29. HR-tools Supply Payments Logistics CRM Support TRADITIONAL DATA STRUCTURE RDBMS RDBMS RDBMSRDBMS RDBMS RDBMS
  30. 30. SHIFT TOWARDS SYSTEMS OF ENGAGEMENT
  31. 31. Users Engaging With DevicesUsers Engaging With Users Devices Engaging With Devices SYSTEMS OF ENGAGEMENT
  32. 32. SHIFT TOWARDS SYSTEMS OF ENGAGEMENT You are here! Data volume
  33. 33. SYSTEMS OF RECORD Relational Database Model Structured Pre-computed Based on rigid rules SYSTEMS OF ENGAGEMENT NoSQL Database Model Highly Flexible Real-Time Queries Highly Contextual
  34. 34. http://bigdata-blog.com/nosql-definition
  35. 35. Polyglot Persistence
  36. 36. Polyglot Persistence • Different types of data in different ways • Take advantage of strengths of different databases http://martinfowler.com/bliki/PolyglotPersistence.html
  37. 37. Polyglot Persistence Functionality Database type Shopping Cart Rapid session reads / writes Key-value store Orders / Product Catalog Frequent reads Document Customer social graph Recommendation Graph http://www.jamesserra.com/archive/2015/07/what-is-polyglot-persistence/
  38. 38. Data Storage and Business Rules Execu5on Data Mining and Aggrega5on Applica'on Graph Database Cluster Neo4j Neo4j Neo4j Ad Hoc Analysis Bulk Analy'c Infrastructure Hadoop, EDW … Data Scien'st End User Databases Rela5onal NoSQL Hadoop
  39. 39. Added Complexity
  40. 40. Challenge #1: Data Sync
  41. 41. MIGRATE ALL DATA MIGRATE SUBSET DUPLICATE SUBSET Non-Graph Queries Graph Queries Graph Queries Non-Graph Queries All Queries Rela3onal Database Graph Database Application Application Application Non Graph Data All Data ?
  42. 42. Course Catalog Browse & Search • Good use-case for a document database
  43. 43. https://github.com/johnymontana/CourseCatalog
  44. 44. Course Catalog Browse & Search • Good use-case for a document database
  45. 45. Graph Based Recommendations
  46. 46. Graph Based Recommendations
  47. 47. Graph Based Recommendations
  48. 48. ?
  49. 49. Option 1: APOC Procedures
  50. 50. https://github.com/neo4j-contrib/neo4j-apoc-procedures
  51. 51. Java Stored Procedures User-defined procedures are written in Java, deployed into the database, and called from Cypher. http://neo4j.com/docs/developer-manual/current/#procedures
  52. 52. Built-in Procedures
  53. 53. Built-in Procedures
  54. 54. User-defined Procedures https://github.com/neo4j-examples/neo4j-procedure-template
  55. 55. Apoc Procedures https://github.com/neo4j-contrib/neo4j-apoc-procedures
  56. 56. THERE’S A PROCEDURE FOR THAT https://github.com/neo4j-contrib/neo4j-apoc-procedures
  57. 57. https://github.com/neo4j-contrib/neo4j-apoc-procedures/blob/master/readme.adoc
  58. 58. https://neo4j-contrib.github.io/neo4j-apoc-procedures/ #_loading_data_from_web_apis_json_xml_csv
  59. 59. ?
  60. 60. https://docs.mongodb.com/ecosystem/tools/http-interfaces/
  61. 61. Option 2: Neo4j Doc Manager
  62. 62. Neo4j Doc Manager • Automatically sync documents from MongoDB to Neo4j • Convert documents into graph model http://neo4j.com/developer/mongodb/
  63. 63. Neo4j Doc Manager Automatically sync documents from MongoDB to Neo4j
  64. 64. Document to property graph { "session": { "title": "12 Years of Spring: An Open Source Journey", "abstract": "Spring emerged as a core open source project in early 2003 and evolved to a broad portfolio of open source projects up until 2015." }, "topics": [ "keynote", "spring" ], "room": "Auditorium", "timeslot": "Wed 29th, 09:30-10:30", "speaker": { "name": "Juergen Hoeller", "bio": "Juergen Hoeller is co-founder of the Spring Framework open source project.", "twitter": "https://twitter.com/springjuergen", "picture": "http://www.springio.net/wp-content/ uploads/2014/11/juergen_hoeller-220x220.jpeg" } }
  65. 65. { "session": { "title": "12 Years of Spring: An Open Source Journey", "abstract": "Spring emerged as a core open source project in early 2003 and evolved to a broad portfolio of open source projects up until 2015." }, "topics": [ "keynote", "spring" ], "room": "Auditorium", "timeslot": "Wed 29th, 09:30-10:30", "speaker": { "name": "Juergen Hoeller", "bio": "Juergen Hoeller is co-founder of the Spring Framework open source project.", "twitter": "https://twitter.com/springjuergen", "picture": "http://www.springio.net/wp-content/ uploads/2014/11/juergen_hoeller-220x220.jpeg" } } Document to property graph
  66. 66. https://github.com/neo4j-contrib/neo4j_doc_manager
  67. 67. Demo
  68. 68. Using Data Relationships for Recommendations Content-based filtering Recommend items based on what users have liked in the past Collaborative filtering Predict what users like based on the similarity of their behaviors, activities and preferences to others Movie Person Person RATED SIMILARITY rating: 7 value: .92
  69. 69. Challenge #2: Operations
  70. 70. http://neo4j.com/developer/docker/
  71. 71. https://hub.docker.com/_/neo4j/ docker pull neo4j Official Neo4j Docker Image
  72. 72. https://www.docker.com/products/docker-compose
  73. 73. docker-compose.yml
  74. 74. Resources
  75. 75. http://neo4j.com/blog/couchbase-jdbc-integrations-neo4j-3-0/ Couchbase + Neo4j integration
  76. 76. http://neo4j.com/developer/ Neo4j Developer Resources
  77. 77. Neo4j Integrations http://neo4j.com/developer/integration/ Neo4j + MongoDB http://neo4j.com/developer/mongodb/ Neo4j Doc Manager https://github.com/neo4j-contrib/neo4j_doc_manager
  78. 78. Questions? @lyonwj will@neo4j.com

×