Talk at QCon London. Based on an article in InfoWorld (https://www.infoworld.com/article/3254777/application-development/3-common-pitfalls-of-microservices-integrationand-how-to-avoid-them.html). Source code available: https://github.com/flowing/flowing-retail/tree/master/payment-rest Recording available: https://youtu.be/O2-NHptllKQ

1. 3 Common Pitfalls in
Microservice Integration and
How to Avoid Them
@berndruecker

2. Berlin, Germany
bernd.ruecker@camunda.com
@berndruecker
Bernd Ruecker
Co-founder and
Developer Advocate of
Camunda

3. REST, SOAP,
Cloud, Saas,
Microservices, SCS,
FaaS, Serverless,
…

4. Distributed systems

5. Distributed systems
Challenges of
asynchronicity
Distributed
Transactions
Communication
is complex

6. https://www.infoworld.com/article/3254777/application-development/
3-common-pitfalls-of-microservices-integrationand-how-to-avoid-them.html

7. Some
Service
Some
Service
Some
Service
Some
Service
Some
Service
Some
Service
Some
Service
Failure will happen.
Accept it!
But keep it local!
Be resilient.

8. Let‘s start with a simple example
Credit
Card
Payment
REST

9. Live hacking
https://github.com/flowing/flowing-retail/blob/master/payment-
rest/src/main/java/io/flowing/retail/payment/port/resthacks/PaymentRestHacksControllerV2.java

10. Circuit
Breaker
Photo by CITYEDV, available under Creative Commons CC0 1.0 license.

11. Fail fast
is important

12. Fail fast
is important
but not enough!

13. Photo by Tookapic, available under Creative Commons CC0 1.0 license.

14. „There was an error
while sending your
boarding pass“

15. Check-in
Web-UI
Me
Current situation

16. Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Current situation

17. Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Current situation

18. Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Current situation
Circuit
breaker

20. Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Current situation – the bad part

21. Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Current situation – the bad part

22. Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Current situation – the bad part
Stateful
Retry

24. We are having some technical
difficulties and cannot present you
your boarding pass right away.
But we do actively retry ourselves, so
lean back, relax and we will send it
on time.

25. Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Possible situation – much better!

26. Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Possible situation – much better!
Stateful
Retry

27. Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Stateful
Retry
Possible situation – much better!
The failure
never leaves
this scope!

28. Persist thing
(Entity, Document, Actor, …)
State machine or
workflow engine
Typical
concerns
DIY = effort,
accidental
complexity
Complex, proprietary,
heavyweight, slow,
don‘t scale,
developer adverse
Scheduling, Versioning,
operating, visibility,
scalability, …
Handling
State

29. Workflow engines,
state machines
It is
relevant
in modern
architectures

30. CADENCE
Silicon valley
has recognized
Workflow engines,
state machines

31. CADENCE
Workflow engines,
state machines

32. CADENCE
also at scale
Workflow engines,
state machines

33. CADENCE
for todays demo
Workflow engines,
state machines

34. Live hacking
https://github.com/flowing/flowing-retail/blob/master/payment-
rest/src/main/java/io/flowing/retail/payment/port/resthacks/PaymentRestHacksControllerV3.java

35. Payment
Now you have a state machine!
Credit
Card
REST

36. has to implement
Retry
has to implement
Idempotency
Client Service Provider

37. We are processing your payment.
Do not leave this page.
And for god sake – do not reload!
It is a business
problem
anyway!

38. We are processing your payment.
Do not leave this page.
And for god sake – do not reload!
It is a business
problem
anyway!
We are currently processing your request.
Don‘t worry, it will happen safely –
even if you loose connection.
Feel free to reload this page any time!

39. Distributed systems introduce complexity you have to tackle!
Credit
Card
Payment
REST

40. Distributed
systems

41. It is impossible to
differentiate certain
failure scenarios.
Independant of
communication style!
Service
Provider
Client

42. Distributed systems introduce complexity you have to tackle!
Credit
Card
Payment
REST

43. Distributed systems introduce complexity you have to tackle!
Credit
Card
Payment
REST

44. BPMN
Business Process
Model and Notation
ISO Standard

45. Workflows live within
service boundaries
Credit
Card
Payment
REST

46. Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Workflows live within
service boundaries

47. Manigfold architecture options
https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91

48. Manigfold architecture options
https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91

49. Manigfold architecture options
https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91

50. Manigfold architecture options
https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91

51. generateBoardingPass
HTTP 200 OK
HTTP 202 ACCEPTED
Check-in
A synchronous response is possible in the
happy case, otherwise it is switched to
asynchronous processing.

52. „The customer wants a synchronous response“
Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
„Eh – no!“

53. Synchronous communication
is the crystal meth of
distributed programming
Todd Montgomery and Martin Thompson
in “How did we end up here” at GOTO Chicago 2015

54. Challenges of
asynchronicity

55. Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Asynchronous communication
You need to
monitor
timeouts

56. Check-in
Barcode
Generator
Web-UI
Me
Output
Mgmt
Remember…
The failure
never leaves
this scope!

57. Workflow…

58. Workflow…

59. has to implement
Timeout, Retry
has to implement
Idempotency
Client Service Provider

60. Who uses a message bus?

61. Who has no problems
operating a message bus?
Dead messages | No context | Inaccesible payload | Hard to redeliver |
Home-grown message hospitals | …

62. Manigfold architecture options
https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91

63. Manigfold architecture options
https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91

64. Manigfold architecture options
https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91

65. Distributed
Transactions
Distributed
Transactions

66. Distributed
systems

67. Distributed transactions using compensation *
Compensation

68. Eventual consistency
Temporarily
inconsistent state
But only
temporarily!

69. Live hacking
https://github.com/flowing/flowing-retail/blob/master/payment-
rest/src/main/java/io/flowing/retail/payment/port/resthacks/PaymentRestHacksControllerV6.java

70. has to implement
Timeout, Retry,
Compensation
has to offer
Compensation
has to implement
Idempotency
Client Service Provider

71. has to implement
Timeout, Retry,
Compensation
has to offer
Compensation
has to implement
Idempotency
Client Service ProviderDon‘t forget
about state

72. Before mapping processes
explicitly with BPMN, the truth was
buried in the code and nobody
knew what was going on.
Jimmy Floyd, 24 Hour Fitnesse
„

73. Event-driven example also available
InventoryPaymentOrder ShippingCheckout Monitor
https://github.com/flowing/flowing-retail/
Human
Tasks
H2 H2

74. Workflows live inside service boundaries

76. Sales-Order & Order-Fulfillment
via Camunda
for every order worldwide
(Q2 2017: 22,2 Mio)

77. # Be aware of complexity of distributed systems
# Know strategies and tools to handle it
e.g. Circuit breaker (Hystrix)
Workflow engine for stateful retry, waiting, timeout
and compensation (Camunda)

78. Thank you!
Meet me at the
Camunda booth!

79. bernd.ruecker@camunda.com
@berndruecker
https://bernd-ruecker.com
https://blog.bernd-ruecker.com
https://github.com/flowing
https://www.infoq.com/articles/
events-workflow-automation With thoughts from http://flowing.io
@berndruecker | @martinschimak
Contact:
Slides:
Blog:
Code:
https://www.infoworld.com/article/3254777/
application-development/
3-common-pitfalls-of-microservices-
integrationand-how-to-avoid-them.html