Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Using BigQuery as a main Big Data solution

1 369 vues

Publié le

My recent slide deck on BigQuery and how we use it in Wego.com

Publié dans : Données & analyses
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ http://1lite.top/ROpLZ ◀ ◀ ◀ ◀
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ http://1lite.top/ROpLZ ◀ ◀ ◀ ◀
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • Soyez le premier à aimer ceci

Using BigQuery as a main Big Data solution

  1. 1. Nikolay Novozhilov Wego.com Using BigQuery as a main Big Data solution
  2. 2. About Wego Wego.com is Asia Pacific and the Middle East’s leading flight/hotel metasearch engine used by millions of travelers. Wego was founded in 2005 in Singapore
  3. 3. Introducing BigQuery Service for interactive analysis of massive datasets (TBs) Query billions of rows: seconds to write, seconds to return Uses a SQL-style query syntax It's a service, accessed by a RESTful API Pay only for what you use Based on internal Google tool - Dremel Column oriented, append only…
  4. 4. Data architecture in Wego ...
  5. 5. Why did we do it? MySQL “Zoo” BigQuery
  6. 6. Why Hadoop is more popular?
  7. 7. My collection of concerns Your data goes to cloud Not open-source, Google can stop the service “Strange” pricing model Hadoop is trending, has bigger community Append only database ???
  8. 8. Costs: storage + cost per query Same fallacy again:  “I want to launch a mom@pop – let’s buy a building”  “I want to build a site – let’s by servers”  “I want big data – let’s build a data-warehouse” Usual concerns:  No realistic estimate upfront  “Fear of running a query”
  9. 9. StackOverflow support 53 minutes!
  10. 10. Append only… Slowly changing dimensions:  daily re-load from MySQL  daily upload from MySQL, keeping history Absolutely necessary updates:  do you really need it?  BigQuery allows to save query to initial table: Your table Query
  11. 11. Actually useful - “Discovery mode”
  12. 12. Actually useful Huge joins REGEXT_MATCH(), … Rich SQL - window functions Nested data
  13. 13. My answer
  14. 14. What is Big Data revolution? There is no difference between big data and small data anymore
  15. 15. Contacts Blog: www.novozhilov.co Email: nik@wego.com
  16. 16. “Yes, Sir, I tired to build an ROI case for our BI project - but I couldn’t access any reliable data!” TimoElliott.com

×