Ce diaporama a bien été signalé.
Le téléchargement de votre SlideShare est en cours. ×

Ubiquitous Solr - A Database's Not-So-Evil Twin: Presented by Ayon Sinha, WalmartLabs

Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité

Consultez-les par la suite

1 sur 22 Publicité
Publicité

Plus De Contenu Connexe

Diaporamas pour vous (20)

Les utilisateurs ont également aimé (14)

Publicité

Similaire à Ubiquitous Solr - A Database's Not-So-Evil Twin: Presented by Ayon Sinha, WalmartLabs (20)

Plus par Lucidworks (20)

Publicité

Plus récents (20)

Ubiquitous Solr - A Database's Not-So-Evil Twin: Presented by Ayon Sinha, WalmartLabs

  1. 1. Ubiquitous Solr - A Database’s not-so-evil Twin Ayon Sinha Data Foundation @WalmartLabs O C T O B E R 1 3 - 1 6 , 2 0 1 6 • A U S T I N , T X
  2. 2. 2 Text Search wow Search Suggestions Search Engine… Lucene… Solr •  Internet and Intranet Search •  Relevance •  Search Suggestions •  Faceting •  Recommendations •  Time series •  Log search •  Geo-spatial search •  Analytics •  Graph search •  Document Store Recommendations Relevance Facets
  3. 3. 3 Overview •  How to scale any data infrastructure with Apache Solr •  Build a high performance and highly available data platform for internal and external users alike •  Walmart’s commitment to open source
  4. 4. 4 About me •  Team lead at the Data Foundation team for the largest retailer and the largest private employer in the world •  Prior to Walmart, worked at startups building recommendation and analytics systems •  And prior to that, was building search applications, recommendations systems and Hadoop based analytics systems for the largest online auction company, ebay, for 6 years •  Have been a manuscript reviewer for Manning publications for 4 years and have helped shape the contents of “Hadoop in Practice” and “Big Data”
  5. 5. 5 About Walmart •  11,000+ Stores in 27 countries •  11 eCommerce sites •  250M customers weekly in stores and online •  Millions of database transactions per day •  Sales, Holidays and massive volume shifts
  6. 6. 6 It starts-up so simple An idea implemented on the LAMP stack
  7. 7. 7 Turns out to be a great idea! Users seem to like the new product
  8. 8. 8 Users REALLY like this.. Higher volume, increased use cases. Quick fix scaling alternatives add some headroom … and complexity
  9. 9. 9 We need more Business Intelligence Business is looking good but source-of-truth data store, not so much …
  10. 10. 10 Scale up (in a hurry) with hardware Least risk. Diminishing returns. What next?
  11. 11. 11 Design to scale out •  Offload queries to Search Engines •  Offload recurring reads to Cache •  Offload analytics to OLAP datastores •  Shard the database … and of course do something to hide the complexity. It is worth it.
  12. 12. 12 The Inspiration Integration tools with a Lucene based search engines are abundant
  13. 13. 13 The “not-so-evil” Twin to protect your Source of Truth DB •  What if a copy of your source-of-truth data is available … Just about anywhere you want it? •  How could you use a search engine to protect and augment your database? –  Redirect queries •  Helps scale by reducing demand for –  database indexing –  database connections –  scarce database resources like memory, storage •  Not-so-evil Twin –  Adding multiple near real-time search adds complexity … and it comes at a cost; but done right, the benefits far outweigh the costs
  14. 14. 14 Our Approach •  Abstract the complexity of managing –  source-of-truth database –  cache coherence –  Search queries –  message bus •  Abstract Connection pool management •  Provide a scalable way to query across shards with full control of Solr schema •  And to analyze big data without affecting real-time systems and isolating individual data domains
  15. 15. 15 From a situation like..
  16. 16. 16 DB, Solr and Hadoop
  17. 17. 17 Sharded DB with Solr
  18. 18. 18 The Eco-system Separation of concerns
  19. 19. 19 The Result Scatter-gather vs Powered by Apache Solr
  20. 20. 20 Lessons learned A Search engine like Apache Solr is… •  not limited to search-based business applications. •  a first class citizen in your persistence technology stack; it complements the SoT database. •  easy to adopt and has all of us as community for support.
  21. 21. 21 The Future •  Symbiotic existence of Solr/Lucene with RDBMS, NoSQL and Big Data systems •  Walmart is committed to be part of the community building it
  22. 22. 22 Questions? Reach us at: •  You can reach me, Ayon Sinha, at: –  asinha@walmartlabs.com –  https://www.linkedin.com/in/ayonsinha •  Jason Sardina, our Lead Persistence Architect –  jsardina@walmartlabs.com •  @WalmartLabs is always hiring the best

×