1. The journey to Native Cloud
Architecture & Microservices
Customized for @Jitta - Mek Srunyu Stittri
Tracing the footsteps of Adrian Cockcroft

2. Agenda
● Background, case study from Netflix
● Continuous Delivery
● Building pillars of frictionless dev and deploy
● Microservices, abstraction and architecture

3. Enterprise late adoption of the cloud

4. Why did Netflix do this
Speed
Innovation
New Ideas
New Products

5. SPEED, is always going to win in the market.
The faster you go the more competitive you are.
Netflix VS Comcast, Apple, Google, Amazon and etc..
What they did : Be really fast, Being disruptive &
Continuous Deployment
Why did Netflix do this

6. What Netflix learned
● Speed wins in the marketplace
● Remove friction from product development (CD)
● High-trust, low process, no handoffs between teams
● Freedom and responsibility culture
● Don’t do your own undifferentiated heavy lifting
● Use simple patterns automated by tooling, Jenkins
● Self service cloud makes impossible things instant

7. Non cloud product deployment

8. Let’s eliminate HW provisioning

9. IaaS Based Product Development

10. Let's eliminate SW provisioning

11. PaaS Based Product Development

12. Go even further… use SaaS

13. What Happened..

14. Continuous Delivery on Cloud

15. Continuous Delivery on Cloud4

16. Build, Release and architecture
Monolithic VS Microservice (CD)

17. Monolithic approach

18. Microservices approach

19. Immutable code, old code remains

20. Independent development streams

21. New code deploys into service groups

22. Non-destructive production updates
● Immutable Code Service Pattern
○ Existing services are unchanged, old code remains in service
○ New code deploys as a new service group
○ No impact to production until traffic routing changes
● A|B Tests, Feature Flags and Version Routing control traffic
○ First users in the test cell are the developers and test engineers
○ A cohort of users is added looking for measurable improvement
○ Finally make the new version the default for everyone, keep
older version for a while

23. How do we get there...
Old way :
● Dev runs code, IT deploys code.
New way:
● Deployment should be automatic, self service API driven.
● Dev runs code + dev deploys code.
● Freedom but also incentive to be responsible.
Culture and trust.

24. Non-destructive production updates
● Make your assumptions explicit
● Extrapolate trends to the limit
● Listen to non-customers
● Follow developer adoption not IT spending
● Map evolution of products to services to
utilities
● Re-organize your teams for speed of execution

25. Non-destructive production updates
● Business logic isolation in stateless micro-services
● Immutable code with instant rollback
● Auto-scaled capacity and deployment updates
● Distributed across availability zones and regions
● De-normalize single function NoSQL data stores
● See over 40 NetflixOSS projects at - netflix.githb.com
● Netflix engineering blog - techblog.netflix.com

26. Mircroservice.. what is it ?
See http://en.wikipedia.org/wiki/Domain-driven_design for discussion of bounded contexts
A definition of : Microservices
Loosely coupled service oriented
architecture with bounded contexts

27. Scaling Continuous Delivery
Monolithic
● Devs book a train ticket
● Everyone runs the monolith
● Queue for the next train
● Coordination chat session
● Need to learn deploy process
● Copy code to existing servers
● Few concurrent versions
● Tens of monolithic updates/day
maximum
● Roll-forward only
● Done - is release to production
Microservices
● Every team has their own build
● Dev runs their own microservice
● No waiting, no meetings
● API call to update prod timeline
● Automated hands-off deploy
● Immutable code on new servers
● Unlimited concurrent versions
● 100s of independent updates
● Roll-back in seconds
● Done - is retired from production

28. Separate components using Micro-services
● Invert Conway’s Law - teams own service groups and backend
stores
● One “Verb” per single function micro-service, size does not
matter
● One developer independently produces a micro-service
● Each micro-service is its own build, avoid trunk conflicts
● Deploy in a container: Tomcat, AMI, or Docker etc..
● Stateless business logic. Cattle not pets (dispensable)
● Stateful cached data access layer can use ephemeral instances
●

29. Microservices Development Architecture
Client libraries
● Even if you start with a raw protocol, a client side driver is the end-state
● Best strategy is the own your own client libraries from the start
Multi-threading and non-blocking calls
● Reactive model RxJava uses Observable to hide concurrently cleanly
● Netty can be used to get non-blocking IO speedup over Tomcat container
Circuit breakers - See Fluxcapacitor.com for code
● NetflixOSS Hystrix, Turbine, Latency Monkey, Ribbon/Karyon
● Also look at Finagle/Zipkin from Twitter

30. Microservice Datastores
Book : Refactoring Databases
● SchemaSpy to examine schema structure
● Denormalization into one datasource per table or materialized view
Polyglot Persistence
● Use a mixture of database technologies, behind REST data access layers
● See NetflixOSS Storage Tier as a Service HTTP (staash.com) for MySQL and C*
CAP - Consistent or Available when Partitioned
● Look at Jepsen torture tests for common systems aphyr.com/tags/jepsen
● Also look at Finagle/Zipkin from Twitter

31. Microservice examples

32. Monitoring...
Monitoring for Microservices
Great speed comes with great
responsibility and code ownership

33. Challenges with Continuous Delivery
High rate of change
● Code pushes can cause floods of new instances and metrics
● Short baseline for alert thresholds analysis - everything looks unusual
Ephemeral Configurations
● Short lifetimes make it hard to aggregate historical views
● Hand-tweaked monitoring tools take too much work to keep running
Microservices with complex calling patterns
● End-to-end request flow measurements are very important
● Request flow visualizations get overwhelmed

34. Monitoring Continuous Delivery
● Changes are smaller but more frequent
● Individual changes are more likely to be
broken
● Change are normally deployed y developers
● Feature flags are used to enable new code
● Instant detection and rollback matters much
more
●

35. Continuous Monitoring

36. Continuous Monitoring
Whoops! I didn’t mean that
Reverting...
Not cool if it takes 5 mins to detect a failure and another 5 mins to fix
No one notices if it takes 5 seconds to detect and 5 more to rollback

37. Continuous Monitoring

38. NetflixOSS Hystrix / Turbine
circuit breaker monitoring system

39. Latest SaaS Monitoring Products

40. Summary
● Speed wins in the Market
● Remove friction from development
● High trust, low process
● Freedom and responsibility culture
● Leverage SAAS solutions to do heavy lifting
● Automate simple patterns in tools
● Microservices for speed and high-availability

41. Resources
Adrian Cockroft
Adrian’s Blog http://perfcap.blogspot.com
Adrian’s Slides http://www.slideshare.net/adriancockcroft/qcon-new-york-speed-and-scale
NetFlix Hystrix / Turbine
http://techblog.netflix.com/2012/12/hystrix-dashboard-and-turbine.html
See over 40 NetflixOSS projects at netflix.github.com
Monitoring tools
VividCortex.com / Boundary.com