Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Pinot: Realtime Distributed OLAP datastore

565 820 vues

Publié le

Pinot is a realtime distributed OLAP datastore, which is used at LinkedIn to deliver scalable real time analytics with low latency. It can ingest data from offline data sources (such as Hadoop and flat files) as well as online sources (such as Kafka). Pinot is designed to scale horizontally.

Publié dans : Technologie
  • Effective Love Spell To Get Your Ex Back My advise reveal you all to contact Dr happy the best on love spell at happylovespell2@gmail.com, here is my relationship story. About a month ago I got back to my ex(after 2 and half years separated).Everything was ok and she was the most adorable woman in the world. she spoke about wedding ceremony, how many kids we would have, said that our relationship was the most important thing in her life and she was 100% sure about it…Then she traveled to work in other country for 3 weeks. she changed his behavior completely. she disappeared every night, was saying she had to do some work and would talk to me later, but disappeared the rest of the day and night… no explanation, when I asked why she was disappearing every night, she stopped answering me. she’s ignoring me since last week, April 15. she ignored even my birthday, that was April 16. Not even a message. I wrote to her a few times trying to understand what happened, she read but just ignored me.I feel so bad cause I couldn’t stop contacting and trying to understand what happened.i loved her so much and wanted to get her back, then i was told to contact happylovespell2@gmail.com who can help out with a love spell to reunite your relationship back, so instantly do contact him and give him a try and trust me he did an urgent effective love spell for me and with in 24hours promise he made to me, my woman was back, today i testify to you all that you should seek and contact Dr happy for his spell result is genuine sure and guarantee unite and get your ex back save your marriage/relationship problem now, You can also know more about his spell Blogs site.http://happylovespel.blogspot.com.ng/ Email him now on.. happylovespell2@gmail.com If you want to talk for more info or consultation. Whatsapp/cal +2348133873774 For more assurance and spell guarantee, visit his website...happylovespell2.webnode.com/
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • Effective Love Spell To Get Your Ex Back My advise reveal you all to contact Dr happy the best on love spell at happylovespell2@gmail.com, here is my relationship story. About a month ago I got back to my ex(after 2 and half years separated).Everything was ok and she was the most adorable woman in the world. she spoke about wedding ceremony, how many kids we would have, said that our relationship was the most important thing in her life and she was 100% sure about it…Then she traveled to work in other country for 3 weeks. she changed his behavior completely. she disappeared every night, was saying she had to do some work and would talk to me later, but disappeared the rest of the day and night… no explanation, when I asked why she was disappearing every night, she stopped answering me. she’s ignoring me since last week, April 15. she ignored even my birthday, that was April 16. Not even a message. I wrote to her a few times trying to understand what happened, she read but just ignored me.I feel so bad cause I couldn’t stop contacting and trying to understand what happened.i loved her so much and wanted to get her back, then i was told to contact happylovespell2@gmail.com who can help out with a love spell to reunite your relationship back, so instantly do contact him and give him a try and trust me he did an urgent effective love spell for me and with in 24hours promise he made to me, my woman was back, today i testify to you all that you should seek and contact Dr happy for his spell result is genuine sure and guarantee unite and get your ex back save your marriage/relationship problem now, You can also know more about his spell Blogs site.http://happylovespel.blogspot.com.ng/ Email him now on.. happylovespell2@gmail.com If you want to talk for more info or consultation. Whatsapp/cal +2348133873774 For more assurance and spell guarantee, visit his website...happylovespell2.webnode.com/
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • Effective Love Spell To Get Your Ex Back My advise reveal you all to contact Dr happy the best on love spell at happylovespell2@gmail.com, here is my relationship story. About a month ago I got back to my ex(after 2 and half years separated).Everything was ok and she was the most adorable woman in the world. she spoke about wedding ceremony, how many kids we would have, said that our relationship was the most important thing in her life and she was 100% sure about it…Then she traveled to work in other country for 3 weeks. she changed his behavior completely. she disappeared every night, was saying she had to do some work and would talk to me later, but disappeared the rest of the day and night… no explanation, when I asked why she was disappearing every night, she stopped answering me. she’s ignoring me since last week, April 15. she ignored even my birthday, that was April 16. Not even a message. I wrote to her a few times trying to understand what happened, she read but just ignored me.I feel so bad cause I couldn’t stop contacting and trying to understand what happened.i loved her so much and wanted to get her back, then i was told to contact happylovespell2@gmail.com who can help out with a love spell to reunite your relationship back, so instantly do contact him and give him a try and trust me he did an urgent effective love spell for me and with in 24hours promise he made to me, my woman was back, today i testify to you all that you should seek and contact Dr happy for his spell result is genuine sure and guarantee unite and get your ex back save your marriage/relationship problem now, You can also know more about his spell Blogs site.http://happylovespel.blogspot.com.ng/ Email him now on.. happylovespell2@gmail.com If you want to talk for more info or consultation. Whatsapp/cal +2348133873774 For more assurance and spell guarantee, visit his website...happylovespell2.webnode.com/
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • MY HUSBAND IS BACK WITH THE GREAT HELP OF DR SAM LOVE SPELL I'm so excited my broken Marriage has been restored my ex husband is back after he left me and our 2kids for another woman. i was so happy to met Dr. Sam Blogs https://supremespells.blogspot.com how he help many people to bring there Lover back so i contact him to help me too. that was how Dr. Sam help me to bring my husband back.. A big thank to you Dr. Sam because I never thought my ex Husband will be back to me so quickly with your spell. You are the best and world greatest. if you are here and you need your Ex Lover back or your husband moved to another woman, do not cry anymore, contact this powerful spell caster now. Here’s his contact: Email him at: Supremespellcast@gmail.com Or Supremespellcast@yahoo.com WhatsApp Or Call: +2347087462033
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • Effective Love Spell To Get Your Ex Back My advise reveal you all to contact Dr happy the best on love spell at happylovespell2@gmail.com, here is my relationship story. About a month ago I got back to my ex(after 2 and half years separated).Everything was ok and she was the most adorable woman in the world. she spoke about wedding ceremony, how many kids we would have, said that our relationship was the most important thing in her life and she was 100% sure about it…Then she traveled to work in other country for 3 weeks. she changed his behavior completely. she disappeared every night, was saying she had to do some work and would talk to me later, but disappeared the rest of the day and night… no explanation, when I asked why she was disappearing every night, she stopped answering me. she’s ignoring me since last week, April 15. she ignored even my birthday, that was April 16. Not even a message. I wrote to her a few times trying to understand what happened, she read but just ignored me.I feel so bad cause I couldn’t stop contacting and trying to understand what happened.i loved her so much and wanted to get her back, then i was told to contact happylovespell2@gmail.com who can help out with a love spell to reunite your relationship back, so instantly do contact him and give him a try and trust me he did an urgent effective love spell for me and with in 24hours promise he made to me, my woman was back, today i testify to you all that you should seek and contact Dr happy for his spell result is genuine sure and guarantee unite and get your ex back save your marriage/relationship problem now, You can also know more about his spell Blogs site.http://happylovespel.blogspot.com.ng/ Email him now on.. happylovespell2@gmail.com If you want to talk for more info or consultation. Whatsapp/cal +2348133873774 For more assurance and spell guarantee, visit his website...happylovespell2.webnode.com/
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici

Pinot: Realtime Distributed OLAP datastore

  1. Pinot Kishore Gopalakrishna Tuesday, August 18, 15
  2. Agenda • Pinot @ LinkedIn - Current • Pinot - Architecture • Pinot Operations • Pinot @ LinkedIn - Future Tuesday, August 18, 15
  3. WVMP Tuesday, August 18, 15
  4. Slice and Dice Metrics Tuesday, August 18, 15
  5. Pinot @ LinkedIn Customers Members Internal tools Tuesday, August 18, 15
  6. • 100B documents • 1B documents ingested per day • 100M queries per day • 10’s of ms latency • 30 tables in prod, 250 * 3 std app nodes Pinot @ LinkedIn Tuesday, August 18, 15
  7. Key features SQL-like interface Columnar storage and indexing Real-time data load Tuesday, August 18, 15
  8. (S)QL: Filters and Aggs SELECT count(*) FROM companyFollowHistoricalEvents WHERE entityId = 121011 AND 'day' >= 15949 AND 'day' <= 15963 AND paid = 'y’ AND action = 'stop' Tuesday, August 18, 15
  9. (S)QL: Group By SELECT count(*) FROM companyFollowHistoricalEvents WHERE entityId = 121011 AND 'day' >= 15949 AND 'day' <= 15963 AND paid = 'y’ GROUP BY action Tuesday, August 18, 15
  10. (S)QL: ORDER BY and LIMIT SELECT * FROM companyFollowHistoricalEvents WHERE entityId = 121011 AND entityId = 1000 AND action = 'start' ORDER BY creationTime DESC LIMIT 1 Tuesday, August 18, 15
  11. Whats not supported • JOIN: unpredictable performance • NOT A SOURCE OF TRUTH • Mutation Tuesday, August 18, 15
  12. Pinot • Data flow • Query Execution • How to use/operate • Pinot @ LinkedIn - Future Tuesday, August 18, 15
  13. Broker Helix Real time Historical Kafka Hadoop Pinot Architecture Queries Raw Data Tuesday, August 18, 15
  14. Pinot • Pinot segments Tuesday, August 18, 15
  15. Pinot Segment layout: Columnar storage Tuesday, August 18, 15
  16. Pinot Segment layout: Sorted Forward Index Tuesday, August 18, 15
  17. Pinot Segment layout: Other techniques • Indexes: Inverted index, Bitmap, RoaringBitmap • Compression: Dictionary Encoding, P4Delta • Multi Valued columns, skip lists, • Hyperloglog for unique • T-digest for Percentile, Quantile Tuesday, August 18, 15
  18. Data aware pre-computation Star tree Index Tuesday, August 18, 15
  19. Pinot • Query Execution Tuesday, August 18, 15
  20. Pinot Query Execution: Distributed Servers S1 S3 S2 S1 S3 S2 Helix Brokers Tuesday, August 18, 15
  21. Pinot Query Execution: Distributed Servers 1.Query S1 S3 S2 S1 S3 S2 Helix Brokers Tuesday, August 18, 15
  22. Pinot Query Execution: Distributed Servers 1.Query S1 S3 S2 S1 S3 S2 Helix 2. Fetch routing table from HelixBrokers Tuesday, August 18, 15
  23. Pinot Query Execution: Distributed Servers 1.Query S1 S3 S2 S1 S3 S2 Helix 2. Fetch routing table from HelixBrokers 3. Scatter Request Tuesday, August 18, 15
  24. Pinot Query Execution: Distributed Servers 1.Query S1 S3 S2 S1 S3 S2 Helix 2. Fetch routing table from HelixBrokers 3. Scatter Request 4. Process Request & send response Tuesday, August 18, 15
  25. Pinot Query Execution: Distributed Servers 1.Query S1 S3 S2 S1 S3 S2 Helix 2. Fetch routing table from HelixBrokers 3. Scatter Request 4. Process Request & send response 5. Gather Response Tuesday, August 18, 15
  26. Pinot Query Execution: Distributed Servers 1.Query S1 S3 S2 S1 S3 S2 Helix 2. Fetch routing table from HelixBrokers 3. Scatter Request 4. Process Request & send response 5. Gather Response 6. Return Response Tuesday, August 18, 15
  27. Pinot Query Execution: Single Node Architecture EXECUTION ENGINE INVERTED INDEX BITMAP INDEX COLUMN FORMAT PLANNER Tuesday, August 18, 15
  28. Pinot Query Execution: Single Node Architecture SELECT campaignId, sum(clicks) FROM Table A WHERE accountId = 121011 AND 'day' >= 15949 GROUP BY campaignId account Id daycampaign Id click Filter Operator Projection Operator Aggregation Group by Operator Combine Operator Pinot Segments Data sources Matching doc ids campaignId,Click tuple Tuesday, August 18, 15
  29. Pinot • Operations Tuesday, August 18, 15
  30. Cluster Management: Deployment Helix Brokers Servers • Brokers and Servers register themselves in Helix • All servers start with no use case specific configuration Controller Tuesday, August 18, 15
  31. On boarding new use case Helix Brokers Servers XLNT XLNT XLNT Create Table command Controller XLNT XLNTTag Servers TableName Brokers 3 XLNT_T1 1 Tuesday, August 18, 15
  32. Segment Assignment Servers S3 S2 S1 Upload Segment S2 S1 S3 S2 S1 S3 Helix Brokers Copies TableName 2 XLNT_T1 Controller Tuesday, August 18, 15
  33. • AUTO recovery mode: Automatically redistribute segments on failure/addition of new nodes • Custom mode: Run in degraded mode until node is restarted/replaced. Pinot - Fault tolerance/Elasticity Tuesday, August 18, 15
  34. Pinot vs Druid Druid Pinot Architecture Realtime + Offline, Realtime only Realtime + Offline Realtime only -> consistency is hard and schema evolution/Bootstrap is hard Inverted Index Always On all columns, Fixed Configurable on per column basis Allows trade off between scanning v/s inverted index + scanning. More data can be fit in given memory size Data organization N/A Sorts data Organizing data provides speed/better compression and removes the need for inverted index Smart pre- materialization N/A star-tree Allows trade off between latency and space Query Execution Layer Fixed Plan Split into Planning and execution Smart choices can be made at runtime based on metadata/query. Tuesday, August 18, 15
  35. • Documentation & tooling • In progress - consistency among real time replicas. • Improve cost to serve - leverage SSD, partial pre materialization • ThirdEye - Business Metrics Monitoring Pinot - Future Tuesday, August 18, 15
  36. Thank You 30 Tuesday, August 18, 15

×