At Netflix, we provide an API that supports the content discovery, sign-up, and playback experience on thousands of device types that millions use around the world every day. As our user base and traffic has grown by leaps and bounds, we are continuously evolving this API to be flexible, scalable, and resilient and enable the best experience for our users. In this talk, I gave an overview of how and why the Netflix API has evolved to where it is today and how we make it resilient against failures while keeping it flexible and nimble enough to support continuous A/B testing.

1. Evolving the
Netflix API
Katharina Probst
Engineering Manager, API
October 2015

2. What is Netflix?

3. > 1000 Devices

4. Is it significant?
❏ Peak
downstream
traffic in the
US is almost
35%.
❏ Almost 70 Million subscribers worldwide and
growing
Source: http://www.sandvine.com/news/global_broadband_trends.asp

5. We’re going
global!

6. Source: https://help.netflix.com/en/node/14164
Recent additions:
Spain, Portugal, Italy
Current availability

7. Netflix
Originals

8. Do we need a
Netflix API?

9. API
Personali-
zation
Engine
User
Info
Ratings
Similar
Movies
A/B Test
Engine
….

10. Uses
❏ Discovery
❏ Signup
❏ Playback
❏ Internal teams
only
API

11. Goals
❏ Flexibility
❏ Resiliency
❏ Scalability
❏ Excellent tools
API

12. Goals
❏ Flexibility
❏ Resiliency
❏ Scalability
❏ Excellent tools
API

13. Lots of devices, lots of variety

14. Different interaction models

15. And just to make things a little more
interesting….
❏ A/B tests
❏ profiles
❏ localization

16. What we felt
we had
What we
needed

17. ❏ Reduce network chattiness
❏ Support device optimizations
❏ Enable faster development for internal users

18. Local MethodRemote API
GET
/users/{user_id}/lists
apiGateway
.getLists(userId)

20. Discrete HTTP requests pay network tax repeatedly

21. Single, optimized request; pay network tax once

22. Single, optimized request; pay network tax once
Client data
assembly logic
pushed to server

23. Add server-side scripting capability
❏ Enable independent development & device
optimization
❏ Profit

24. ❏ UI (script) changes can happen
independently
❏ Script changes can be pushed to running
servers, so decoupled from API push
schedule
❏ Server+UI changes usually involve API team
Impact on velocity and collaboration

25. RxJava Hystrix
JavaServiceLayer
Mid-tier
Services
UI
Teams
Client Server
Internet
Application
/tv/home
API
Team
Service
Teams

26. ELB Zuul
Mid-tier
Services
Scriptable
Backend
Scriptable
Backend
+
API Layer

27. Goals
❏ Flexibility
❏ Resiliency
❏ Scalability
❏ Excellent tools
API

28. https://github.com/Netflix/Hystrix
resilience patterns for distributed
sys

29. Hystrix Primer
❏ Protection from and control over
latency and failure from dependencies
❏ Stop cascading failures in a complex
distributed system
❏ Fail fast and rapidly recover
❏ Fall back and gracefully degrade

30. Personalization
Engine
Similar
Movies
Movie
Metadata
Ratings User Info
Instant
Queue
A/B Test
Engine
API

31. Personalization
Engine
Similar
movies
Movie
Metadata
Ratings User Info
Instant
Queue
A/B Test
Engine
API

32. API
Personalization
Engine
Similar
movies
Movie
Metadata
Ratings User Info
Instant
Queue
A/B Test
Engine
Beware
Cascading
Failure!

33. Personalization
Engine
Similar
Movies
Movie
Metadata
Ratings User Info
Instant
Queue
A/B Test
Engine
API

34. Personalization
Engine
Similar
Movies
Movie
Metadata
Ratings User Info
Instant
Queue
A/B Test
Engine
Fallback
Response
Local
Fallback
Avoids
Cascading
Failure!
API

35. Personalization
Engine
Similar
Movies
Movie
Metadata
Ratings User Info
Instant
Queue
A/B Test
Engine
Fallback
Response
Use FIT
to test
such
failures
API

36. Goals
❏ Flexibility
❏ Resiliency
❏ Scalability
❏ Excellent tools
API

37. Autoscaling & Capacity Management
http://nflx.it/1LvqLUi

38. AWS Controls Reactive, does not
scale up fast enough

39. Fine-grained Control with Scryer
Complements AWS Controls
❏ Faster scale-up, improved cost
❏ Use reactive policy for organic scale down

40. Goals
❏ Flexibility
❏ Resiliency
❏ Scalability
❏ Excellent tools
API

41. Run 1% of your traffic on the new
code and see how it does

42. ❏ Errors: 2xx, 4xx, 5xx
❏ latency
❏ network
❏ busy threads
❏ load
❏ ...
So you’ve run a canary. Now what?
Control Canary

44. Successful canary
red/black push

45. Continuous Delivery
http://techblog.netflix.com/2015/09/moving-from-asgard-to-spinnaker.html

46. Quickly see status of all clusters
http://techblog.netflix.com/2015/09/moving-from-asgard-to-spinnaker.html

47. Script Management

49. Deployment & Ops

50. Deployment & Ops

51. Deployment & Ops

52. Real-time analysis
http://www.slideshare.net/g9yuayon/qcon-talk-on-netflix-mantis-a-stream-processing-system
Submit a query, see requests in real time.

53. Looking ahead - current challenges
❏ Breaking up the monolith
❏ Script isolation
❏ Thin client libraries
❏ New interaction models

54. Looking ahead
Source: http://techcrunch.com/2014/03/08/success-reality-and-the-myth-of-up-and-to-the-right/

55. Looking ahead
❏ Breaking up the monolith
❏ Script isolation
❏ Thin client libraries
❏ New interaction models

56. ● > 900 active
endpoints
● ~ 30 client libraries
● 78 thread pools
● high memory usage
Breaking up the
monolith

57. Script isolation & node
❏ Groovy scripts run as
part of API process
❏ UI teams would like to
use other languages
(in particular node.js) API remote
service layer
Service client
libraries
UI/device
scripts (node)
Falcor
var response = model.get("todos[0..2]
['name','done']");

58. Thin client libraries
❏ Many client libraries
contain a lot of
business logic and
have a lot of
dependencies
❏ Move business logic
and dependencies to
server
API remote
service layer
Service client
libraries
UI/device
scripts (node)
Falcor

59. Looking ahead
❏ Breaking up the monolith
❏ Script isolation
❏ Thin client libraries
❏ New interaction models

60. New interaction models
❏ request/response
❏ request/stream
❏ fire-and-forget
❏ event subscription
❏ channel
API remote
service layer
Service client
libraries
UI/device
scripts (node)
Falcor
http://reactivesocket.io

61. In the beginning...