TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
Pnuts
1. PNUTS: Yahoo!’s Hosted Data Serving Platform B.F. Cooper, R. Ramakrishnan, U. Srivastava, A. Silberstein, P. Bohannon, H. Jacobsen, N. Puz, D. Weaver and R. Yerneni Yahoo! Research Seminar Presentation for CSE 708 by Ruchika Mehresh Department of Computer Science and Engineering 22 nd February, 2011
2.
3.
4. PNUTS Data Storage and Retrieval Features Data and Query Model System Architecture Consistency (Yahoo! Message Broker) Query Processing Experiments Recovery Structure Future Work
5.
6. PNUTS Data Storage and Retrieval Features Data and Query Model System Architecture Consistency (Yahoo! Message Broker) Query Processing Experiments Recovery Structure Future Work
7.
8. PNUTS Data Storage and Retrieval Features Data and Query Model System Architecture Consistency (Yahoo! Message Broker) Query Processing Experiments Recovery Structure Future Work
10. PNUTS Data Storage and Retrieval Features Data and Query Model System Architecture Consistency (Yahoo! Message Broker) Query Processing Experiments Recovery Structure Future Work
11.
12.
13. PNUTS Data Storage and Retrieval Features Data and Query Model System Architecture Consistency (Yahoo! Message Broker) Query Processing Experiments Recovery Structure Future Work
14.
15.
16.
17.
18. PNUTS Data Storage and Retrieval Features Data and Query Model System Architecture Consistency (Yahoo! Message Broker) Query Processing Experiments Recovery Structure Future Work
19.
20. PNUTS Data Storage and Retrieval Features Data and Query Model System Architecture Consistency (Yahoo! Message Broker) Query Processing Experiments Recovery Structure Future Work
21.
22.
23.
24. PNUTS Data Storage and Retrieval Features Data and Query Model System Architecture Consistency (Yahoo! Message Broker) Query Processing Experiments Recovery Structure Future Work
30. PNUTS Data Storage and Retrieval Features Data and Query Model System Architecture Consistency (Yahoo! Message Broker) Query Processing Experiments Recovery Structure Future Work
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48. Consistency model Time v. 1 v. 2 v. 3 v. 4 v. 5 v. 7 Generation 1 v. 6 v. 8 Current version Stale version Stale version Read
49. Consistency model Time v. 1 v. 2 v. 3 v. 4 v. 5 v. 7 Generation 1 v. 6 v. 8 Read up-to-date Current version Stale version Stale version
50. Consistency model Time v. 1 v. 2 v. 3 v. 4 v. 5 v. 7 Generation 1 v. 6 v. 8 Read ≥ v.6 Current version Stale version Stale version Read-critical(required version):
51. Consistency model Time v. 1 v. 2 v. 3 v. 4 v. 5 v. 7 Generation 1 v. 6 v. 8 Write Current version Stale version Stale version
52. Consistency model Time v. 1 v. 2 v. 3 v. 4 v. 5 v. 7 Generation 1 v. 6 v. 8 Write if = v.7 ERROR Current version Stale version Stale version Test-and-set-write(required version)
53. Consistency model Time v. 1 v. 2 v. 3 v. 4 v. 5 v. 7 Generation 1 v. 6 v. 8 Write if = v.7 ERROR Current version Stale version Stale version Back Mechanism: per record mastership
54. What is PNUTS? Parallel database Geographic replication Structured, flexible schema Hosted, managed infrastructure E 75656 C A 42342 E B 42521 W C 66354 W D 12352 E F 15677 E E 75656 C A 42342 E B 42521 W C 66354 W D 12352 E F 15677 E A 42342 E B 42521 W C 66354 W D 12352 E E 75656 C F 15677 E
55. Storage units Routers Tablet controller REST API Clients Message Broker Detailed architecture Data-path components
56. Storage units Routers Tablet controller REST API Clients Local region Remote regions YMB Detailed architecture
57. Accessing data SU SU SU Get key k 1 2 Get key k 3 Record for key k 4 Record for key k
58. Bulk read SU SU SU Scatter/ gather server 1 {k 1 , k 2 , … k n } 2 Get k 1 Get k 2 Get k 3
59. Range queries Router Apple Avocado Banana Blueberry Canteloupe Grape Kiwi Lemon Lime Mango Orange Strawberry Tomato Watermelon Storage unit 1 Storage unit 2 Storage unit 3 Grapefruit…Pear? Grapefruit…Lime? Lime…Pear? MIN-Canteloupe SU1 Canteloupe-Lime SU3 Lime-Strawberry SU2 Strawberry-MAX SU1 SU1 Strawberry-MAX SU2 Lime-Strawberry SU3 Canteloupe-Lime SU1 MIN-Canteloupe
60. Updates Write key k Sequence # for key k Sequence # for key k Write key k SUCCESS Write key k Routers Message brokers 1 2 Write key k 7 8 SU SU SU 3 4 5 6
![Motivation ,[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)