Are you getting the most out of your data platform? The technologies you choose are important, but even more so is how you put them into practice. Part philosophy and part pragmatic reality, I'll dive into our thinking at Netflix on technology selection and trade-offs, challenging everything (constructively), providing building blocks and paved paths, staffing, and more. I'll also talk through our tech stack, which includes many big data technologies (e.g. Hadoop, Spark, and Presto), traditional BI tools (e.g. Teradata, MicroStrategy, and Tableau), and custom tools / services (e.g. our big data portal and API). My hope (and expectation!) is that you'll leave with an arsenal of new ideas on the best way to get things done.

1. 1
Elevating Your Data Platform
Kurt Brown

3. • 1/3+ Internet traffic (NA / peak)
• 100+ Million hours per day
• 65+ Million members / 50+ countries
• Outside the US – streaming only

4. 500 Billion Events / Day

6. S3
Stin
g
Data Platform

7. Who chooses the technologies?

8. Why these?
Expertise
Scale
Business
Needs Differentiating
Value
Integration
Contribution
CostLegacy
Maintenance
Future
Community

9. Philosophy

10. The North Star

11. Netflix Expense Policy
“Act in Netflix’s Best Interest”

12. Impact vs. Perfection

13. Question Everything

14. Tell vs. Ask

15. Does it really matter?

16. Misuse of DBAs

17. Development & Deployment Flow
Best
Practices Consultation
Components
Tools
Automation
Clean-up

18. Whose Data Platform is it?

19. Whose Data Platform is it?

20. Visibility

24. What’s mine is yours

25. General OSS
• Pooled goodness
• Manage own destiny
• Vendor relationships
• Conflicting agendas

26. Netflix OSS
• Meet Netflix needs
• Shared direction
• Recruiting & Retention
• Code quality
• Contributions

27. Data Platform OSS

28. Should you open source?
• You get value
• Others would get value
• Will invest some time
• Bonus tip: Open source “upfront”

29. Buy or Build

30. Vendors / Startups
• Sweet Spot
• Maturity
• Business Critical
• Truly Differentiating
• Open Source 

31. Beware of Icebergs

32. The paved path

33. “Inventing” the wheel

34. Tidy Packages & Building Blocks

39. Get your head in the cloud

40. Cost

41. Cost

42. Elasticity
• Grow naturally
• Spikes
• Reservations
• Spot

43. Services
• EMR, SQS, S3,…
• Ready for action
• Focus!

44. Redundancy / Global Presence
• Multi-DC
• International

45. Why not?

46. Photo credit: Ron Wise via Flickr – http://www.flickr.com/photos/available-light/1372534662
Under CC BY 2.0 License – http://creativecommons.org/licenses/by/2.0

47. Cloud Mindset
• Self-service
• Distributed systems
• Red / black deployments
• ….

48. Photo credit: Sids1 via Flickr – http://www.flickr.com/photos/sids1/5338140209
Under CC BY 2.0 License – http://creativecommons.org/licenses/by/2.0

50. Cost?

51. Netflix Culture Deck
Google Search: “Netflix Culture Deck”