Taylor Wicksell and Tom Gianos at SpringOne Platform 2019
•
2 j'aime•734 vues
Spring at Netflix YouTube: https://youtu.be/mln3_o6qlBo
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Taylor Wicksell and Tom Gianos at SpringOne Platform 2019
Similaire à Taylor Wicksell and Tom Gianos at SpringOne Platform 2019 (20)
Plus de VMware Tanzu
Plus de VMware Tanzu (20)
Dernier
Dernier (20)
Taylor Wicksell and Tom Gianos at SpringOne Platform 2019
- 2. Taylor Wicksell — Senior Software Engineer since 2013 — Java Platform Team at Netflix — 10th year of attending SpringOne SPRING @ NETFLIX
- 4. Netflix Java Platform — Small number of engineers — Own the Dev Experience for Java — Accelerate delivery of applications — Ready applications for production — Our mission is Leverage! SPRING @ NETFLIX
- 5. Our Product — Application Generators — Documentation & Support — Libraries • Discovery via Eureka • Configuration • Service to Service communication • Metrics, Tracing, Security, Logging — Originally in-house frameworks — As of 2019, implemented via Spring SPRING @ NETFLIX
- 6. Why Spring? SPRING @ NETFLIX
- 10. Developers need to Developers choose — Prototype faster — Deploy applications to the open Internet — Deliver to desktops, phones, embedded devices — Authorize access to sensitive data — REST, gRPC, GraphQL — MySQL, Redis, ElasticSearch, CockroachDB, Firebase — JPA, Jooq, JDBI, Envers — Kafka, Kinesis, AMQ, Sentry SPRING @ NETFLIX
- 11. SPRING @ NETFLIX
- 12. Benefits to our Developers Benefits to the Platform Team — Spring Boot Starters are easily composable — Production ready features • Spring Security • Spring Cloud • Micrometer — Unifying abstractions • Spring's Environment • Spring Security • Spring Data • Spring Cloud Sleuth — Ready for Java 11 and beyond — Huge ecosystem — Great documentation — Most features "just work" — Google and StackOverflow SPRING @ NETFLIX
- 13. SPRING @ NETFLIX
- 15. Spring Applications vs Legacy Platform SPRING @ NETFLIX Java Applications Spring Legacy
- 16. Spring Boot Applications in Production 100 200 300 400 Dec 2018 Jan 2019 Feb 2019 Mar 2019 Apr 2019 May 2019 Jun 2019 Jul 2019 Aug 2019 Sep 2019 300+
- 17. So what did we learn? SPRING @ NETFLIX — Stick with Spring Abstractions — Empower users to find their own solutions — Platform features must have toggles — Partnership is key!
- 18. SPRING @ NETFLIX
- 19. Spring Boot Netflix In Production Tom Gianos How the Netflix Big Data Platform leverages Spring Boot to enable data analytics at scale SPRING @ NETFLIX
- 20. SPRING @ NETFLIX SPRING @ NETFLIX How is my AB test performing? - Engineer How many members do we have in country X? - Management I need data to train my model - Data Scientist Data Warehouse (S3) > 100 PB Data Warehouse Services Compute Clusters Metadata Management, Data Ingestion, etc Spark, Presto, etc Orchestration Services Scheduler, Job Execution, etc Data API Python Client Web App InsightsServices
- 21. Big Data Platform Spring Service Example ● Distributed job orchestration service ● Executes > 250k jobs per day ● Jobs span a variety of dimensions ○ Short vs long running ○ Ad-hoc vs Scheduled SLA ○ Local execution vs. Cluster execution ● Two Major Components ● Web Server ● Agent ● Majority of code in Netflix OSS ○ https://netflix.github.io/genie/ ○ https://github.com/Netflix/genie ● Internally wrapped in Spring Boot Netflix SPRING @ NETFLIX
- 22. SPRING @ NETFLIX SPRING @ NETFLIX Compute Admins Register Available Resources Add new clusters Remove old clusters Make new version of spark available Promote new version of spark to default Job Request Run a query Execute some custom ETL Return dataset Submit and Monitor
- 23. SPRING @ NETFLIX
- 24. SPRING @ NETFLIX
- 25. SPRING @ NETFLIX
- 26. Spring to the Rescue — Administration • Spring Boot Actuator — Aurora MySQL • Spring Data JPA • Flyway — AWS (S3, SNS, etc) • Spring Cloud AWS — Documentation • Spring Rest Docs — E-Mail • Spring Mail — gRPC • Custom configuration in OSS • Spring Boot Netflix gRPC internally — Metrics • Micrometer — REST API • Spring MVC • Spring Hateoas — Retries • Spring AOP • Spring Retry — Security • Spring Security — State Machine • Spring State Machine — Zookeeper • Spring Cloud Zookeeper • Spring Integration Zookeeper SPRING @ NETFLIX
- 27. SPRING @ NETFLIX Deploying OSS Internally — AutoConfigurations are invaluable • Define default beans in OSS • Override if necessary internally • Pick up Spring Boot Netflix provided beans where necessary • Metrics Registry • Discovery • Zookeeper • Security — Property Bindings and Profiles • Different profiles activated based on deployment dimensions • Environment (test vs prod), Stack, Laptop, CI server, etc. • Spring Boot Netflix provide sensible defaults but allow us to override
- 28. Thank You MONTH 1, 2019