Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2RLYvFy. Andrew McVeigh explains how transitioning to a microservice architecture went for a top-tier video service & also a major games studio. Operating and evolving such significant systems reveals a number of challenges: monitoring and root cause analysis, cascading failures and scaling for major events are just some. McVeigh discusses how to avoid common pitfalls when working with microservices. Filmed at qconsf.com. Andrew McVeigh is VP Architecture at Hulu. He is an experienced technical leader & architect, with a background in many different languages, platforms, and technologies. He's worked successfully at the highest level across a number of domains including gaming, investment banking, digital broadcasting, speech research, and telecoms.

1. Microservices
From Winning the War
…To Keeping the Peace
Andrew McVeigh
QConSF October 2018

2. InfoQ.com: News & Community Site
• Over 1,000,000 software developers, architects and CTOs read the site world-
wide every month
• 250,000 senior developers subscribe to our weekly newsletter
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• 2 dedicated podcast channels: The InfoQ Podcast, with a focus on
Architecture and The Engineering Culture Podcast, with a focus on building
• 96 deep dives on innovative topics packed as downloadable emags and
minibooks
• Over 40 new content items per week
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
microservices-pitfalls-lessons

3. Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
Presented at QCon San Francisco
www.qconsf.com

4. Background
A bit about Andrew’s work...
◎ Many different domains
◎ Trading & risk systems
◎ PhD on software components
◎ Riot Games
◎ Hulu
2

5. Microservices
Work!
But make sure you prepare for the challenges...
3

6. 1. Microservices in Gaming
2. Microservices for Video
3. Believe the Hype
4. Believe the Challenges
4

7. 1.
Microservices in
Gaming
A microservice architecture
@ scale
5

8. League of Legends
◎ Here you have a list of items
◎ And some text
◎ But remember not to overload your slides
with content
Your audience will listen to you or read the
content, but won’t do both.
6

9. @ Scale
7

10. Gaming Particulars
◎ Low latency
◎ Need to match up players = shared state
◎ Rapid development cycles
◎ Lots of engineers working on 1 game
8

11. 1A.
Winning the War
Evolving from a monolith
9

12. Evolving the Architecture
◎ 2009: Large service monolith
○ Matchmaking, game selection, inventory etc
◎ 2012: Started evolving to microservices
○ http://bit.ly/evolving-league
10

13. Front End Box
League of Legends
11
platform.war
Distributed in-memory cache
MySQL
Matchmaking Game Server Mgr
platform.warplatform.warplatform.war platform.warplatform.war
MySQL

14. Microservices for New Features
12
Platform
platform.war
Distributed in-memory cache
MySQL
Service
Proxy / API
MySQL
Match Making
tb.jar
Cache

15. REST SWAGGER
Dashboards Service Discovery Config Service
Standard Infrastructure
13
Client of Service
Metrics
Client REST
library
S/W Load
Balancer
Service
Metrics
Server REST
library
HTTP Server

16. 1B.
Keeping the Peace
Held back by remains of the
monolith
14

17. Not Quite Free of the Monolith...
15
Platform
● Inventory
● Stats
● Summoner details
Distributed in-memory cache
MySQL
MySQL
Microservice
Cache

18. Ouch - Hitting the Wall
16

19. If We Could Redo?
◎ Decouple state completely
○ Inventory service
○ Catalog service
○ Runes service
◎ Socialize to get prioritization
◎ Simplify infrastructure
○ Config System too “clever”
○ Too much “smarts” in fat libraries
17

20. 2.
Microservices for
Internet Video
Hundreds of tiny pieces...
18

21. Hulu
◎ 2016: Full microservices architecture
○ Evolved / replaced existing VOD architecture
○ Live TV
○ 20m+ total subs
○ 1m+ live subs
◎ 15 month development!
○ 800+ microservices
http://bit.ly/hulu-landscape
19

22. Video System Particulars
◎ Lots of caching to support browsing
◎ TV show metadata needed everywhere
◎ Real-time playback to support live TV
◎ Lots of integration (billing, ads etc)
20

23. Infrastructure for Microservices
21
Donki PaaS
AppsGithub Jenkins
Apps
Apps
Redis
Cluster
MySQL
Cluster
Provisioning System
Apps
Apps
VMs
Load
Balancers
DNS
Statsd
Graphite

24. Microservice Ownership
22
Playback team Browse team Recording team Etc...

25. Data Distribution Pattern
23
Master
Cache CacheCacheCache
Ingestion + Admin
Reading
Pub-sub /
read-through

26. Scaling for Launch Day
24
Browse Queries Per Second
College kids falling asleep
with XBOX on!

27. 2B.
Keeping the Peace
Every issue gets magnified...
25

28. Arch / Ops Dashboard
26
Login Browse & Search Playback Ads DVR … etcPlayback

29. http://bit.ly/hulu-scaling
Scaling For Growth
27

30. Circuit Breakers avoid Firestorms
28
A B C
X
YY
X
A B C
Z

31. Cross-Cutting Requirements
29

32. Cloud versus DC
◎ Could target cloud or DC
○ But no elasticity, must overprovision
◎ Abstracting cloud & DC
== Lowest common denominator
◎ Hard to do proper blue-green in DC
30

33. 3.
Believe the Hype
The many benefits of microservices
31

34. They Actually Work!
◎ Ownership & Independence
◎ Development velocity
◎ Operational & Development scaling
32

35. They Actually Work!
◎ Granular deployment
◎ Evolution
◎ Organizational alignment
33

36. 4.
Believe the Challenges
Standing on the shoulders of
constantly improving infrastructure...
34

37. Common CICD + Operating Env
Runtime Infra
35
Build + CICD
Operational Infra
Deploy &
Rollback
Jenkins
Spinnaker / Harness
Terraform
K8s, istio
Stackdriver
Cloud!

38. Cloud Approach
Preferred approach
◎ Pick one cloud provider (per workflow?)
◎ Consider costs early
◎ Multi-region, multi-account on day 1
Cloud has so many advantages over DC
◎ Elasticity
◎ Easy environments (blue-green)
◎ Better shared services (db, queues etc)
36

39. Circuit Breakers by Default!
37

40. API Gateways
38
Gateway
ServiceService Envoy
API Docs

41. Platforms
For things that need holistic treatment
… but still allow self-service
◎ Load testing
◎ Billing
◎ Browse caching
◎ A / B testing
◎ UI layout
39

42. Takeaways
Microservices offer many benefits
◎ Isolation & independence
◎ Granular deployment, scaling & evolution
Use infra to protect against common issues
◎ Full CICD
◎ Infrastructure as code
◎ Circuit breakers to prevent firestorms
◎ Istio to help with monitoring + more
◎ Cloud elasticity FTW
40

43. Thanks for Listening!
Any questions?
andrew.mcveigh@gmail.com
41
AMA
Decomposing the Monolith
Tues 2:55pm Boardroom C

44. 42

45. Scaling For Growth
43But look out!!!

46. Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
microservices-pitfalls-lessons