Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Maintaining the Netflix Front Door - Presentation at Intuit Meetup

4 721 vues

Publié le

This presentation goes into detail on the key principles behind the Netflix API, including design, resiliency, scaling, and deployment. Among other things, I discuss our migration from our REST API to what we call our Experienced-Based API design. It also shares several of our open source efforts such as Zuul, Scryer, Hystrix, RxJava and the Simian Army.

Publié dans : Technologie, Formation
  • DOWNLOAD FULL eBOOK INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookeBOOK Crime, eeBOOK Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici

Maintaining the Netflix Front Door - Presentation at Intuit Meetup

  1. 1. Maintaining the Front Door to Netflix Daniel Jacobson @daniel_jacobson http://www.linkedin.com/in/danieljacobson http://www.slideshare.net/danieljacobson
  2. 2. Global Streaming Video for TV Shows and Movies
  3. 3. More than 48 Million Subscribers More than 40 Countries
  4. 4. Netflix Accounts for >34% of Peak Downstream Traffic in North America Netflix subscribers are watching more than 1 billion hours a month
  5. 5. Netflix Accounts for >6% of Peak Upstream Traffic in North America Netflix subscribers are watching more than 1 billion hours a month
  6. 6. Team Focus: Build the Best Global Streaming Product Three aspects of the Streaming Product: • Non-Member • Discovery • Streaming
  7. 7. The Netflix API - Background
  8. 8. Netflix API
  9. 9. Netflix API Requests by Audience At Launch In 2008 Netflix Devices Open API Developers
  10. 10. Netflix API
  11. 11. Netflix API Requests by Audience From 2011 Netflix Devices Open API Developers
  12. 12. Current Emphasis of Netflix API Netflix Devices
  13. 13. Netflix API : Key Responsibilities • Broker data between services and Devices • Provide features and business logic • Maintain a resilient front-door • Scale the system • Maintain high velocity • Provide detailed insights into the system health
  14. 14. Netflix API : Key Responsibilities • Broker data between services and Devices • Provide features and business logic • Maintain a resilient front-door • Scale the system • Maintain high velocity • Provide detailed insights into the system health
  15. 15. APIs Do Lots of Things!
  16. 16. Data Gathering Data Formatting Data Delivery Security Authorization Authentication System Scaling Discoverability Data Consistency Translations Throttling Orchestration APIs Do Lots of Things! These are some of the many things APIs do.
  17. 17. Data Gathering Data Formatting Data Delivery Security Authorization Authentication System Scaling Discoverability Data Consistency Translations Throttling Orchestration APIs Do Lots of Things! These three are at the core. All others ultimately support them.
  18. 18. Definitions • Data Gathering – Retrieving the requested data from one or many local or remote data sources • Data Formatting – Preparing a structured payload to the requesting agent • Data Delivery – Delivering the structured payload to the requesting agent
  19. 19. Meanwhile… There are two players in APIs
  20. 20. API Provider API Consumer
  21. 21. API Provider PROVIDES API Consumer CONSUMES Traditional API Interactions
  22. 22. API Provider PROVIDES EVERYTHING API Consumer CONSUMES WHAT IS PROVIDED Everything means, API Provider does: • Data Gathering • Data Formatting • Data Delivery • (among other things) Traditional API Interactions
  23. 23. Why do most API providers provide everything? • API design tends to be easier for teams closer to the source • Centralized API functions makes them easier to support • Many APIs have a large set of unknown and external developers
  24. 24. Why do most API providers provide everything? • API design tends to be easier for teams closer to the source • Centralized API functions makes them easier to support • Many APIs have a large set of unknown and external developers
  25. 25. Data Gathering Data Formatting Data Delivery API Consumer API Provider Separation of Concerns To be a better provider, the API should address the separation of concerns of the three core functions
  26. 26. Data Gathering Data Formatting Data Delivery API Consumer Don’t care how data is gathered, as long as it is gathered API Provider Care a lot about how the data is gathered Separation of Concerns
  27. 27. Data Gathering Data Formatting Data Delivery API Consumer Don’t care how data is gathered, as long as it is gathered Each consumer cares a lot about the format for that specific use API Provider Care a lot about how the data is gathered Only cares about the format to the extent it is easy to support Separation of Concerns
  28. 28. Data Gathering Data Formatting Data Delivery API Consumer Don’t care how data is gathered, as long as it is gathered Each consumer cares a lot about the format for that specific use Each consumer cares a lot about how payload is delivered API Provider Care a lot about how the data is gathered Only cares about the format to the extent it is easy to support Only cares about delivery method to the extent it is easy to support Separation of Concerns
  29. 29. Because of our separation of concerns, the Netflix API team is enabled to focus on different charters
  30. 30. Brokering Data to 1,000+ Device Types
  31. 31. Screen Real Estate
  32. 32. Controller
  33. 33. Technical Capabilities
  34. 34. One-Size-Fits-All API Request Request Request
  35. 35. Courtesy of South Florida Classical Review
  36. 36. Resource-Based API vs. Experience-Based API
  37. 37. Resource-Based Requests • /users/<id>/ratings/title • /users/<id>/queues • /users/<id>/queues/instant • /users/<id>/recommendations • /catalog/titles/movie • /catalog/titles/series • /catalog/people
  38. 38. OSFA API RECOMME NDATIONS MOVIE DATA SIMILAR MOVIES AUTH MEMBER DATA A/B TESTS START- UP RATINGS Network Border Network Border
  39. 39. RECOMME NDATIONS MOVIE DATA SIMILAR MOVIES AUTH MEMBER DATA A/B TESTS START- UP RATINGS OSFA API Network Border Network Border SERVER CODE CLIENT CODE
  40. 40. RECOMME NDATIONS MOVIE DATA SIMILAR MOVIES AUTH MEMBER DATA A/B TESTS START- UP RATINGS OSFA API Network Border Network Border DATA GATHERING, FORMATTING, AND DELIVERY USER INTERFACE RENDERING
  41. 41. Experience-Based Requests • /ps3/homescreen
  42. 42. JAVA API Network Border Network Border RECOMME NDATIONS MOVIE DATA SIMILAR MOVIES AUTH MEMBER DATA A/B TESTS START- UP RATINGS Groovy Layer
  43. 43. RECOMME NDATIONSA ZXSXX C CCC MOVIE DATA SIMILAR MOVIES AUTH MEMBER DATA A/B TESTS START- UP RATINGS JAVA API SERVER CODE CLIENT CODE CLIENT ADAPTER CODE (WRITTEN BY CLIENT TEAMS, DYNAMICALLY UPLOADED TO SERVER) Network Border Network Border
  44. 44. RECOMME NDATIONSA ZXSXX C CCC MOVIE DATA SIMILAR MOVIES AUTH MEMBER DATA A/B TESTS START- UP RATINGS JAVA API DATA GATHERING DATA FORMATTING AND DELIVERY USER INTERFACE RENDERING Network Border Network Border
  45. 45. Netflix API : Key Responsibilities • Broker data between services and Devices • Provide features and business logic • Maintain a resilient front-door • Scale the system • Maintain high velocity • Provide detailed insights into the system health
  46. 46. 1000+ Device Types
  47. 47. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies Reviews A/B Test Engine Dozens of Dependencies
  48. 48. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine
  49. 49. Dependency Relationships
  50. 50. 2,000,000,000 Incoming Requests Per Day to the Netflix API
  51. 51. 30 Distinct Dependent Services for the Netflix API
  52. 52. ~500 Dependency jars Slurped into the Netflix API
  53. 53. 14,000,000,000 Netflix API Outbound Calls Per Day to those Dependent Services
  54. 54. 0 Dependent Services with 100% SLA
  55. 55. 99.99% = 99.7%30 0.3% of 2B = 6M failures per day 2+ Hours of Downtime Per Month
  56. 56. 99.99% = 99.7%30 0.3% of 2B = 6M failures per day 2+ Hours of Downtime Per Month
  57. 57. 99.9% = 97%30 3% of 2B = 60M failures per day 20+ Hours of Downtime Per Month
  58. 58. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine
  59. 59. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine
  60. 60. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine
  61. 61. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine
  62. 62. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine
  63. 63. Circuit Breaker Dashboard
  64. 64. Call Volume and Health / Last 10 Seconds
  65. 65. Call Volume / Last 2 Minutes
  66. 66. Successful Requests
  67. 67. Successful, But Slower Than Expected
  68. 68. Short-Circuited Requests, Delivering Fallbacks
  69. 69. Timeouts, Delivering Fallbacks
  70. 70. Thread Pool & Task Queue Full, Delivering Fallbacks
  71. 71. Exceptions, Delivering Fallbacks
  72. 72. Error Rate # + # + # + # / (# + # + # + # + #) = Error Rate
  73. 73. Status of Fallback Circuit
  74. 74. Requests per Second, Over Last 10 Seconds
  75. 75. SLA Information
  76. 76. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine
  77. 77. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine
  78. 78. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine
  79. 79. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine Fallback
  80. 80. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine Fallback
  81. 81. Netflix API : Key Responsibilities • Broker data between services and Devices • Provide features and business logic • Maintain a resilient front-door • Scale the system • Maintain high velocity • Provide detailed insights into the system health
  82. 82. Netflix API : Requests Per Month - 5 10 15 20 25 30 35 RequestsinBillions 50x growth in 18 months
  83. 83. AWS Cloud
  84. 84. Netflix API : Requests Per Month - 5 10 15 20 25 30 35 RequestsinBillions
  85. 85. Autoscaling
  86. 86. Autoscaling
  87. 87. Scryer : Predictive Auto Scaling Not yet…
  88. 88. Typical Traffic Patterns Over Five Days
  89. 89. Predicted RPS Compared to Actual RPS
  90. 90. Scaling Plan for Predicted Workload
  91. 91. What is Scryer Doing? • Evaluating needs based on historical data – Week over week, month over month metrics • Adjusts instance minimums based on algorithms • Relies on Amazon Auto Scaling for unpredicted events
  92. 92. Results
  93. 93. Results : Load Average Reactive Predictive
  94. 94. Results : Response Latencies Reactive Predictive
  95. 95. Results : Outage Recovery
  96. 96. Results : AWS Costs
  97. 97. Scaling Globally
  98. 98. More than 48 Million Subscribers More than 40 Countries
  99. 99. Zuul Gatekeeper for the Netflix Streaming Application
  100. 100. Zuul * • Multi-Region Resiliency • Insights • Stress Testing • Canary Testing • Dynamic Routing • Load Shedding • Security • Static Response Handling • Authentication * Most closely resembles an API proxy
  101. 101. All of these approaches are designed to prevent failures…
  102. 102. But sometimes the best way to prevent failures is to force them!
  103. 103. I randomly terminate instances in production to identify dormant failures. Chaos Monkey
  104. 104. Chaos Gorilla I simulate an outage of an entire Amazon availability zone.
  105. 105. I simulate an outage in an AWS region. Chaos Kong
  106. 106. I find instances that don’t adhere to best practices. Conformity Monkey
  107. 107. I extend Conformity Monkey to find security violations. Security Monkey
  108. 108. I detect unhealthy instances and remove them from service. Doctor Monkey
  109. 109. I clean up the clutter and waste that runs in the cloud. Janitor Monkey
  110. 110. I induce artificial delays and errors into services to determine how upstream services will respond. Latency Monkey
  111. 111. Netflix API : Key Responsibilities • Broker data between services and Devices • Provide features and business logic • Maintain a resilient front-door • Scale the system • Maintain high velocity • Provide detailed insights into the system health
  112. 112. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine
  113. 113. Dependency Relationships
  114. 114. Testing Philosophy: Act Fast, React Fast
  115. 115. That Doesn’t Mean We Don’t Test
  116. 116. Automated Delivery Pipeline
  117. 117. Cloud-Based Deployment Techniques
  118. 118. Current Code In Production API Requests from the Internet
  119. 119. Single Canary Instance To Test New Code with Production Traffic (around 1% or less of traffic) Current Code In Production API Requests from the Internet
  120. 120. Canary Analysis Automation
  121. 121. Single Canary Instance To Test New Code with Production Traffic (around 1% or less of traffic) Current Code In Production API Requests from the Internet Error!
  122. 122. Current Code In Production API Requests from the Internet
  123. 123. Current Code In Production API Requests from the Internet
  124. 124. Current Code In Production API Requests from the Internet Perfect!
  125. 125. Stress Test with Zuul
  126. 126. Current Code In Production API Requests from the Internet New Code Getting Prepared for Production
  127. 127. Current Code In Production API Requests from the Internet New Code Getting Prepared for Production
  128. 128. Error! Current Code In Production API Requests from the Internet New Code Getting Prepared for Production
  129. 129. Current Code In Production API Requests from the Internet New Code Getting Prepared for Production
  130. 130. Current Code In Production API Requests from the Internet Perfect!
  131. 131. Stress Test with Zuul
  132. 132. Current Code In Production API Requests from the Internet New Code Getting Prepared for Production
  133. 133. Current Code In Production API Requests from the Internet New Code Getting Prepared for Production
  134. 134. API Requests from the Internet New Code Getting Prepared for Production
  135. 135. https://www.github.com/Netflix
  136. 136. Maintaining the Front Door to Netflix Daniel Jacobson @daniel_jacobson http://www.linkedin.com/in/danieljacobson http://www.slideshare.net/danieljacobson

×