Sidecars and a Microservices Mesh - Christian Posta

1. A Microservices Journey
@christianposta

2. Christian Posta
Chief Architect, cloud application development
Twitter: @christianposta
Blog: http://blog.christianposta.com
Email: christian@redhat.com
Slides: http://slideshare.net/ceposta
• Author “Microservices for Java developers”
• Committer/contributor lots of open-source
projects
• Worked with large Microservices, web-scale,
unicorn company
• Blogger, speaker about DevOps, integration,
and microservices

4. Rough path of discussions
today
• Microservices: What, Why, When?
• “Cloud-native” with a Platform
• Microservices frameworks
• Service decomposition and boundaries
• Microservice resilience, routing, and control
@christianposta

5. Microserivces:
What, Why, When
@christianposta

6. “The microservice architectural style is an
approach to developing a single application as
a suite of small services, each running in its
own process and communicating with
lightweight mechanisms, often an HTTP
resource API. These services are built around
business capabilities and independently
deployable by fully automated deployment
machinery.”
A microservices definition

7. • Single, self-contained, autonomous
• Isolated and Resilient to faults
• Faster software delivery
• Own their own data
• Easier to understand individually
• Scalability
• Right technology for the problem
• Test individual services
• Individual deployments
Microservices?
@christianposta

8. • System complexity
• Operational complexity
• Testing is harder across services
• Security
• Hard to get boundaries right (transactions,
APIs, etc)
• Resource overhead
• Network overhead
• Lack of tooling
Drawbacks to microservices
@christianposta

9. Why would one implement a system
as microservices?
@christianposta

10. Pain we may feel…
@christianposta
• Making changes in one place negatively affects
unrelated areas
• Low confidence making changes that don’t break
things
• Spend lots of time trying to coordinate work between
team members
• Structure in the application has eroded or is non-
existant
• We have no way to quantify how long code merges
will take

11. @christianposta
• Development time is slow simply because the project
is so big (IDE bogs down, running tests is slow, slow
bootstrap time, etc)
• Changes to one module force changes across other
modules
• Difficult to sunset outdated technology
• We’ve built our new applications around old
premises like batch processing
• Application steps on itself at runtime managing
resources, allocations, computations
Pain we may feel…

12. Microservices is about optimizing for speed.
@christianposta

13. If change is happening on the
outside faster than on the inside
the end is in sight.
S&P company life expectancy
@christianposta
Jack Welch, former CEO, GE

14. Fortune 500 firms in 1955 vs. 2014;
88% are gone
@christianposta

15. Competitive advantage is transient.
We need to continuously re-invent our
business models to compete and stay
relevant.
We need to continuously innovate.
@christianposta

16. Innovation is admitting we don’t
have all the answers
Mark Schwartz – Former CIO USCIS
@christianposta

17. We need to figure out the right
questions to ask…
Mark Schwartz – Former CIO USCIS
@christianposta

18. How do we do this?
@christianposta
• Identify goals
• Free teams to explore possible solution spaces
• Generate hypothesis
• Design cheap experiments to test hypothesis
• Work in small batches
• Learn from results
• Calibrate investment; rinse, repeat

19. “If I invest $5-$10M in your company and you fail, I have 30
other investments. It’s just a footnote in my investment history.”
https://medium.com/@mattklein123/optimizing-impact-why-i-will-not-start-an-envoy-platform-company-8904286658cb
https://barryoreilly.com/2017/04/06/optimize-to-be-wrong-not-right/
Create options through experiments

20. @christianposta
Learning through build-measure-learn

21. Microservices help us go faster.
@christianposta

22. So do we microservices all the way down?
@christianposta

23. @christianposta
http://blog.hypeinnovation.com/using-the-three-horizons-framework-for-innovation
IT Portfolio management strategies

24. @christianposta
Lean Enterprise: http://shop.oreilly.com/product/0636920030355.do

25. @christianposta
http://blog.hypeinnovation.com/using-the-three-horizons-framework-for-innovation
IT Portfolio management
MVPs, experiments, small apps
(co-locate if you have to write an app)
Product development, initial scale
(co-locate perfectly okay here!! ..
Microserices? possibly…)
Starting to feel the weight of maintenance,
need to shoot for efficiencies, integrate
new approaches to increase revenue
(microservices land)

26. Microservices != good design
AND
Co-location != bad design
@christianposta

27. DON’T optimize for microservices if…
@christianposta
• You’re building a Minimum Viable Product (MVP), testing a
hypothesis
• You’re building a CRUD application
• You system isn’t CRUD, but the business logic not very
complicated
• Your system doesn’t have > 10 people all trying to
coordinate to work on it
• Your application doesn’t need to scale
• You deliver packaged software
• You’re building HPC systems

28. Making “cloud-native” economical
@christianposta

29. We can now assert with confidence that
high IT performance correlates with
strong business performance, helping
to boost productivity, profitability and
market share.
@christianposta
https://puppet.com/resources/whitepaper/2014-state-devops-report

30. High performing IT teams
@christianposta
• …are encouraged to experiment
• …learn from failure
• …work in small batches
• …focus on getting continuous feedback
• …are held to outcomes, not output
• …continuously prioritize and reprioritize based on
cost of delay (http://blackswanfarming.com/cost-of-
delay/)

31. High performing IT teams need these
IT capabilities and practices
@christianposta
• Continuous Integration (build from master)
• Continuous Delivery (automated pipelines)
• Safe, reliable delivery mechanisms
• Modern, scalable, resilient application architectures
• Self-service, on-demand infrastructure
• Automated testing
• Metrics, logs, traces, observability
• Feedback loops
• Security as part of the pipeline

32. @christianposta
https://www.infoq.com/articles/cloud-native-panel
"Cloud native” describes applications, architectures,
platforms/infrastructure, and processes, that together
make it economical to work to in small batches to learn
and reduce uncertainty.

33. • Distributed configuration
• Service Discovery
• Loadbalancing
• Circuit Breakers
• Bulkheading
• Versioning/Routing
• Based on AWS
“Cloud-native” platform
What about non-java?
@christianposta

34. Kubernetes
@christianposta

35. Cluster management
• Distributed configuration
• Service Discovery
• Loadbalancing
• Versioning/Routing
• Deployments
• Scaling/Autoscaling
• Liveness/Health checking
• Self healing
• Logging, Metrics, Tracing
@christianposta

36. @christianposta

37. • Team self service application deployment
• Developer workflow
• Enterprise focused (LDAP, RBAC, Oauth, etc)
• Integrated Docker registry
• Jenkins Pipeline (CI/CD) out of the box
• Build/deployment triggers
• Software Defined Networking (SDN)
• Docker native format/packaging
• CLI/IDE/Web based tooling
OpenShift is a Kubernetes platform
@christianposta

39. @christianposta

40. @christianposta

41. @christianposta

42. @christianposta
• Simple configuration
• Curated dependencies and transitive
dependencies
• Built in metrics, monitoring
• Slim profile for deployment
• Strong communities (spring, vert.x,
microprofile.io)
OpenShift Application Runtimes

43. Use Kubernetes/OpenShift
• Distributed configuration
• Service Discovery
• Loadbalancing
• Versioning/Routing
• Deployments
• Scaling/Autoscaling
• Liveness/Health checking
• Self healing
• Logging, Metrics, Tracing
@christianposta

44. What if we’re already using
things like Spring Cloud and/or
Netflix OSS?!
@christianposta

45. spring-cloud-kubernetes
• DiscoveryClient
• Ribbon integration
• Actuator/Health integrations
• Hystrix/Turbine Dashboard integrations
(kubeflix)
• Zipkin Tracing
• Configuration via ConfigMaps
• Archaius Bridge for dynamic configs
https://github.com/spring-cloud-incubator/spring-cloud-kubernetes

46. Microservices boundaries
@christianposta

48. @christianposta

49. Book checkout / purchase Title Search
Recommendations
Weekly reporting
@christianposta

50. @christianposta

51. • Break things into smaller,
understandable models
• Surround a model and its
“context” with a boundary
• Implement the model in code
or get a new model
• Explicitly map between
different contexts
• Model transactional
boundaries as aggregates
Focus on domain models, not data models
@christianposta

52. Service Cutter: A systemic approach to
service
decomposition
@christianposta
https://servicecutter.github.io

53. @christianposta

54. How do we share information?
• REST, RPC
• Streams/Events(ActiveMQ, JMS, AMQP, STOMP, Kafka,
etc)
• Legacy (SOAP, mainframe, file processing, proprietary)
• Routing, Aggregation, Splitting, Transactions,
Compensations, Filtering, etc.
@christianposta

55. • Small Java library
• 200+ components for integrating systems (bring along only
the ones you use)
• Powerful EIPs (routing, transformation, error handling)
• Distributed-systems swiss-army knife!
• Declarative DSL
• Embeddable into any JVM (EAP, Karaf, Tomcat, Spring
Boot, Dropwizard, Wildfly Swarm, no container, etc)
Apache Camel
@christianposta

56. @christianposta

57. public class OrderProcessorRouteBuilder extends RouteBuilder {
@Override
public void configure() throws Exception {
rest().post(“/order/socks”)
.description(“New Order for pair of socks”)
.consumes(“application/json”)
.route()
.to(“activemq:topic:newOrder”)
.log(“received new order ${body.orderId}”)
.to(“ibatis:storeOrder?statementType=Insert”);
}
Camel REST DSL
@christianposta

58. Microservices resilience, routing,
control
@christianposta

59. Things you must solve for because…
distributed systems
• Service discovery
• Retries
• Timeouts
• Load balancing
• Rate limiting
• Thread bulk heading
• Circuit breaking

60. …continued
• Routing between services (adaptive, zone-aware)
• Deadlines
• Back pressure
• Outlier detection
• Health checking
• Traffic shaping
• Request shadowing

61. …continued
• Edge/DMZ routing
• Surgical / fine / per-request routing
• A/B rollout
• Internal releases / dark launches
• Fault injection
• Stats, metric, collection
• Logging
• Tracing

62. @christianposta
http://bit.ly/application-networking

63. @christianposta
http://bit.ly/application-networking

64. @christianposta
http://bit.ly/application-networking

65. • Netflix Hystrix (circuit breaking / bulk heading)
• Netflix Zuul (edge router)
• Netflix Ribbon (client-side service discovery / load balance)
• Netflix Eureka (service discovery registry)
• Brave / Zipkin (tracing)
• Netflix spectator / atlas (metrics)
“Microservices” patterns
@christianposta

66. @christianposta
http://bit.ly/application-networking

67. But I’m using Spring!
• spring-cloud-netflix-hystrix
• spring-cloud-netflix-zuul
• spring-cloud-netflix-eureka-client
• spring-cloud-netflix-ribbon
• spring-cloud-netflix-atlas
• spring-cloud-netflix-spectator
• spring-cloud-netflix-hystrix-stream
• …..
• ......
• @Enable....150differentThings

68. But I’m using Vert.x!
• vertx-circuit-breaker
• vertx-service-discovery
• vertx-dropwizard-metrics
• vertx-zipkin?
• …..
• ......

69. But I’m using NodeJS!
But I’m using Go!
But I’m using Python!

70. Get the point?

71. @christianposta
https://lyft.github.io/envoy/
Meet Envoy Proxy

72. Sidecar pattern

74. Meet Istio Service Mesh
https://istio.io

75. Quick Demo
https://istio.io/docs/samples/bookinfo.html

76. • Have self-service infrastructure automation?
• Have self-service application automation?
• Have working CI/CD?
• Have health checking, monitoring,
instrumentation?
• Have logging, distributed tracing?
• Able to release services independently?
• Honoring backward and forward
Are you doing microservices?
@christianposta

77. • Number of features accepted
• % of features completed
• User satisfaction
• Feature Cycle time
• defects discovered after deployment
• customer lifetime value (future profit as a result of relationship with the
customer) https://en.wikipedia.org/wiki/Customer_lifetime_value
• revenue per feature
• mean time to recovery
• % improvement in SLA
• number of changes
• number of user complaints, recommendations, suggestions
• % favorable rating in surveys
• % of users using which features
• % reduction in error rates
• avg number of tx / user
• MANY MORE!
Focus on going fast and learning

78. • The hardest part of microservices? Your data
https://developers.redhat.com/blog/2016/08/02/the-hardest-part-about-microservices-your-data/
• Microservices patterns:
circuit breaking with Envoy Proxy
https://developers.redhat.com/blog/2017/05/31/microservices-patterns-with-envoy-sidecar-proxy-
part-i-circuit-breaking/
• Monolith to microservices Part I
https://developers.redhat.com/blog/2017/09/26/low-risk-monolith-microservice-evolution-part/
• Monolith to microservices Part II
https://developers.redhat.com/blog/2017/10/23/low-risk-monolith-microservice-evolution-part-ii/
More material
@christianposta

79. • Download and explore OpenShift
• https://www.openshift.org/minishift/
• Checkout Spring Boot/WildFlySwarm/Vert.x on
OpenShift:
• https://launch.openshift.io
• Reach out to your Red Hat rep to discuss more and/or
get me/my team involved with your initiatives
What next?

81. Thanks!
BTW: Hand drawn diagrams made with Paper by FiftyThree.com @christianposta
Twitter: @christianposta
Blog: http://blog.christianposta.com
Email: christian@redhat.com
Slides: http://slideshare.net/ceposta
Follow up links:
• http://openshift.io
• http://launch.openshift.io
• http://blog.openshift.com
• http://developers.redhat.com/blog
• https://www.redhat.com/en/open-innovation-labs
• https://www.redhat.com/en/technologies/jboss-middleware/3scale
• https://www.redhat.com/en/technologies/jboss-middleware/fuse