Codemotion Rome 2015 - Micro-service systems deliver wonderful adaptability to business needs, easy scalability, and low-risk deployment. What's not to like? You also end up with a system that's hard to understand, measure and predict. Traditional approaches to monitoring simply aren't powerful enough to handle the emergent properties of a system with lots of moving parts. The solution is to apply the scientific method! Anything can be measured. Uncertainty can be reduced, and stability can be an emergent property. We just have to learn the lessons that the natural world can teach us.

1. Measuring Micro-services
Richard Rodger @rjrodger
nearform.com

2. Pulse. Temperature. Blood Pressure.
Micro-service systems have
emergent properties too.

3. nearForm & micro-services
50+ production systems.
The good, the bad, and the ugly.

4. What are micro-services?
Independent processes that
exchange messages.

5. Messages are fundamental.
Message behavior has emergent
properties.

6. Message flow rate.
Easy to measure. Tells you a lot.
Independent of services.

7. Deploy a new micro-service.
Does the new version
break anything?

8. To measure changes to services,
measure changes to message
flow rates.

9. Micro-service message patterns.
What to measure?
Here’s what we’ve found useful…

10. Actor.
A pool of services share message
load evenly (round-robin, say).

11. Subscriber.
Many services all listen for the
same set of message types.

12. Chain.
An initial message causes a chain
of serial message steps.

13. Tree.
An initial message causes a
flowering of child messages.

14. Why?
It's the risk, stupid!
Monoliths put everybody in danger.

15. Risk.
Reduction can be measured.
Leave elimination to Machiavelli.

16. Our "best practices" for risk.
Unit tests; code reviews; standards.
Do we have good measures?

17. We've just made things worse!
Microservices also have
emergent failure modes.

18. Let's attack both sides.
And we'll find a way to connect
both attacks.
Formal Methods
“Best Practices”
Measurement

19. Dynamic measurement.
Measure health of the system.
Exposes unknown unknowns.

20. Formal methods.
Correctness proofs are impractical.
Incomplete execution traces? FTW!
* http://research.microsoft.com/en-us/um/people/lamport/tla/tla.html
TLA+
Leslie Lamport *

21. Invariants.
Some things should never change.
Measure them to make sure!

22. Example.
E-commerce shopping cart.
add-item msgs == sales-tax msgs

23. Be practical!
Finding invariants is hard. Use the
microservice patterns to cheat.

24. Actor.
n actors means each actor sees
1⧸nof the messages.

25. Subscriber.
n subscribers means each sees
n messages.

26. Chain.
n inbound messages over k links
means nk chained messages.

27. Tree.
n inbound messages over k leaves
means nk leaf messages.

28. Look for cause/effect relationships.
These are by design! Validate your
message patterns in production.
“ask not what can go wrong,
ask what must go right…”
Chris Newcombe, AWS*
* http://research.microsoft.com/en-us/um/people/lamport/tla/amazon.html

29. When should you rollback?
Invariants should be the same
before and after deployment.

30. Is the system correct?
Business rules are invariants too!
Express as message relationships.

31. Are you about to be blindsided?
Combine individual indicators to
get a deeper measure of risk.

32. Measure what counts.
Find invariants. Measure them.
Follow: senecajs.org

33. Thank You!
Richard Rodger @rjrodger
nearform.com