Microservices at Netflix have evolved over time from a single monolithic application to hundreds of fine-grained services. While this provides benefits like independent delivery, it also introduces complexity and challenges around operations, testing, and availability. Netflix addresses these challenges through tools like Hystrix for fault tolerance, Eureka for service discovery, Ribbon for load balancing, and RxNetty for asynchronous communication between services.
1. MicroServices at NETFLIX
Best Practices & Tools of the trade
Sudhir Tonse
Manager, Cloud Platform
@stonse
http://linkedin.com/in/sudhirtonse
Nitesh Kant
Platform Architect
@NiteshKant
http://linkedin.com/in/niteshkant
2. ● Old DataCenter
(2008)
● Everything in
one WebApp
(.war)
● AWS Cloud
(2012)
● 100s of Fine
Grained
Services
3. Positives
● Isolation brings better Availability*
● Independent Speed of Delivery (by different
teams)
● Decentralized Governance (DevOps)
4. Challenges
Distributed Systems are inherently Complex
Operational Overhead (100s of services; DevOps model absolutely required)
Service Interface Versioning, Mismatches?
Testing (Need the entire ecosystem to test)
Fan out of Requests -> Increases n/w traffic
24. Netflix IPC Stack (1.0)
A
p
a
c
h
e
H
T
T
P
C
l
i
e
n
t
Eureka (Service Registry)
Server (Karyon)
Apache
Tomcat
Client
H
y
s
t
r
i
x
E
V
C
a
c
h
e
Ribbon
Load
Balancing
Eureka
Integration
Metrics
(Servo)
Bootstrapping (Governator)
Metrics (Servo)
Admin ConsoleHTTP
Eureka Integration
Registration
Fetch Registry
25. Netflix IPC Stack (2.0)
Client (Ribbon 2.0)
Eureka (Service Registry)
Server (Karyon)
Ribbon Transport
Load
Balancing
Eureka
Integration
Metrics
(Servo)
Bootstrapping (Governator)
Metrics (Servo)
Admin Console
HTTP
Eureka Integration
Registration
Fetch Registry
Ribbon
Hystrix
EVCache
R
x
N
e
t
t
y
RxNetty
UDP
TCP
WebSockets
SSE
27. Synchronous Applications
Tomcat
Connector Application code Hystrix
Apache HTTP
Client
Conn 1Thread 1 Thread 1’
Thread
1*
Thread 1’
Conn 2Thread 2 Thread 2’
Thread
2*
Thread 2’
Conn nThread n Thread n’
Thread
n*
Thread n’
……....
Large # of connections / Large # of external dependencies => tons of threads.
*If there isn’t any application driven thread change
28. Asynchronous applications
Application code RxNettyHystrixRxNetty
Eventloop 1 Eventloop 4
Eventloop 1*
Eventloop 4*
*If there isn’t any application driven thread change
“N” connections per eventloop
Request processing in Eventloop
Hystrix used for throttling not for achieving asynchronicity.
Eventloops are shared between In & OUT
29. Asynchronous Applications
Application code RxNettyHystrixRxNetty
Eventloop 1 Eventloop 4
Eventloop 1*
Eventloop 4*
*If there isn’t any application driven thread change
Eventloop 2 Eventloop 3
Eventloop 1*
Eventloop 4*
……....
Eventloop 4 Eventloop 1
Eventloop 1*
Eventloop 4*
# of processors => # of eventloops. No dependence on # of connections
30. Takeaway
MicroServices is a better architecture
compared to Monolithic Apps
However
Beaware of the challenges - Use Best Practices
and battle-tested OSS components