Presented in DDD Greece Meetup #4. Recaps on CQRS, discusses when async messaging should be implemented and shows how.

1. ASYNC MESSAGING in CQRS
PART 1: MASSTRANSIT
DDD GREECE

2. CARGO TRACKER

3. CARGO TRACKER
ABOUT
▸ Eric Evans “Big Blue Book”
▸ Published 2003
▸ Imaginary Shipping Company
▸ Chapter 7

4. CARGO TRACKER
KEY FUNCTIONALITY 1/2
▸ Book a Cargo
▸ Delivery Spec (Origin, Destination + Arrival DeadLine)
▸ (Re-)Assign an Itinerary
▸ matches Delivery Spec
▸ derives from Legs of 1+ Vessel Voyages

5. CARGO TRACKER
KEY FUNCTIONALITY 2/2
▸ Register a Handling Event
▸ Load, Unload etc
▸ Report the:
▸ Transport Status (In Port, On Board Vessel …)
▸ Routing Status (Not Routed, Routed, …)
▸ Next Expected Handling Event
▸ Delivery History

6. CARGO TRACKER
DOMAIN MODEL - CARGO AGGREGATE
▸ Cargo (Root)
▸ HandlingEvent
▸ Methods:
▸ New(DeliverySpec)
▸ AssignItinerary(Itinerary)
▸ ChangeDelivery(DeliverySpec)
▸ RegisterHandlingEvent(HandlingEvent)

7. CARGO TRACKER
DOMAIN MODEL - CARGO AGGREGATE
▸ State:
▸ TransportStatus
▸ RoutingStatus
▸ NextExpected: HandlingEvent
▸ DeliveryHistory
▸ Collection of HandlingEvents

8. CARGO TRACKER
REFACTORING - HANDLING EVENT

9. CARGO TRACKER
REFACTORING - HANDLING EVENT

10. CARGO TRACKER
REFACTORING - DELIVERY HISTORY
▸ “leaves Delivery History with no persistent state”
▸ “no real need to keep it around”
▸ “we could derive Delivery History itself whenever it is
needed to answer some question”

11. CARGO TRACKER
REFACTORING - HANDLING EVENT AGGREGATE
▸ HandlingEvent Aggregate:
▸ no domain logic (anemic)
▸ only repository + event class
▸ repository servicing query needs of other aggregates

12. CQRS RECAP

13. CQRS RECAP
ABOUT
▸ Command-Query Responsibility Segregation
▸ keep separate read (query) and write (command) models
▸ DeliveryHistory: Read/Query
▸ Cargo: Write/Command
▸ practically SRP

14. CQRS RECAP
COMMANDS
▸ BookNewCargo, AssignItinerary etc.
▸ App Layer
▸ CommandHandler
▸ BookNewCargoHandler, AssignItineraryHandler etc.
▸ practically an App Service
▸ SRP

15. CQRS RECAP
EVENTS 1/2
▸ NewCargoBooked, DeliveryChanged etc.
▸ Domain Layer
▸ Emitted by A/R after Command Applying
▸ Dispatched by Command Handler

16. CQRS RECAP
EVENTS 2/2
▸ EventHandler
▸ e.g. NewCargoBookedHandler, DeliveryChangedHandler
▸ App Layer
▸ Typical Handlers:
▸ Aggregate / Bounded Context Integration
▸ wired to another Command Handler
▸ Read Model Update

17. CQRS RECAP
READ MODEL UPDATE 1/3
*
* * * * *
*
* *
* * *
* *
* *
* *
* * *
* * * * *
* * * * * *

18. CQRS RECAP
READ MODEL UPDATE 2/3
1 M
1
M
1
M
1M

19. CQRS RECAP
READ MODEL UPDATE 3/3
*
* * * * *
*
* *
* * *
* *
* *
* *
* * *
* * * * *
* * * * * *

20. CQRS RECAP
MULTIPLE EVENT HANDLERS
*
* * * * *
*
* *
* * *
* *
* *
* *
* * *
* * * * *
* * * * * *
* *
* *
* * *
* * *
* *
*

21. CQRS RECAP
MISCONCEPTIONS 1/2
▸ needs two datastores/databases
▸ must be NoSQL
▸ requires async processing
▸ needs async messaging/queueing framework
▸ consistency only eventual

22. CQRS RECAP
MISCONCEPTIONS 1/2
▸ needs two datastores/databases
▸ must be NoSQL
▸ requires async processing
▸ needs async messaging/queueing framework
▸ consistency only eventual
TRADE OFFS

23. CQRS RECAP
MISCONCEPTIONS 2/2
▸ needs a CQRS framework
▸ works only with event sourcing
▸ applied end-to-end
▸ e.g. CRUD

24. CQRS RECAP
SYNC PROCESSING
▸ single process (e.g. Web App)
▸ decreases availability
▸ if process is UI affects UX
▸ in-process event dispatcher for decoupling
▸ rely on base framework facilities
▸ immediate consistency

25. CQRS RECAP
ASYNC PROCESSING
▸ multiple processes (e.g. WebApp and WorkerApp)
▸ higher availability, scaling out
▸ consistency eventual
▸ typically implemented with async messaging
▸ message bus
▸ in-process event dispatcher allows for applying selectively

26. CQRS RECAP
TYPICAL SYSTEM with CQRS + ASYNC PROCESSING
/ *
-BBB
3
B BB #
/ A A *
3 BB # 3 A
A BC A B A C A C B BC C
B 2 3
AB BCB BC C CB
B B A B B CB
C
B BB #
B 3B C 2 3 A B C A 3 3
. 3 - 3
C
2 B BC C#
/ A A * (
3 A A
A 2 BB C
3 AB#
C A 2 BB# C
B B2A A#
/ A A * )
2 ACB CB
C A 3 3
2CB 3 AB BCB
/A C 3

27. ASYNC
MESSAGING

28. ASYNC MESSAGING
MESSAGE BUS 1/2
▸ frameworks: NServiceBus, MassTransit, Rebus etc.
▸ transports: RabbitMQ, Azure Service Bus etc.
▸ at least-once delivery
▸ FIFO
▸ reliable messaging
▸ automatic transport configuration setup
▸ pub/sub, send/receive

29. ASYNC MESSAGING
MESSAGE BUS 1/2
▸ frameworks: NServiceBus, MassTransit, Rebus etc.
▸ transports: RabbitMQ, Azure Service Bus etc.
▸ at least-once delivery
▸ FIFO
▸ reliable messaging
▸ automatic transport configuration setup
▸ pub/sub, send/receive

30. ASYNC MESSAGING
ASYNC & RELIABLE MESSAGING
ALL=CA .KI AK
N ELDAK A AK AM
N L KE AK ( A AE AK
AM
N L KE AK )
1P D= CA -
NANA ( NANA )
1P D= CA . R
)
-/3IK1KKIK
(
A
-/3IK
=
LC A E AKQ EL 142-.41 ( # , 2B
LC EL IM - A Q MDA .KI AK =
1KKIK EL K=ELA E MDA A AK
LC A E AKQ EL 142-.41 ) # , 2B LC EL IM - A IK
= A Q MDA A AE AK MDA LC LM=QL E MDA NANA
LC A E AKQ A MI
A # EL - / ,
A AK EL I A =BMAK
CAMME C MDA -
LC A E AKQ EL
142-.41 # ,
NANAL = A
AKLELMA M = KILL
KI AK KALM=KML
- = EM # 1 - 41
AAEA

31. ASYNC MESSAGING
MESSAGE BUS 2/2
▸ connection management
▸ persistent sagas/process managers
▸ retrying policy on subscriber/receiver errors
▸ alarm service
▸ (de)serialization

32. ASYNC MESSAGING
CQRS & MESSAGING PATTERNS
Command Handler Receiver
Command Issuer Sender
Event Handler Subscriber
Event Emitter Publisher

33. ASYNC MESSAGING
COMMAND FLOW
1. Process A
A. Issue
B. Send
2. Process B
C. Receive
D. Handle - Create Events

34. ASYNC MESSAGING
EVENT FLOW
1. Process B
A. Emit
B. Publish
2. Process C
C. Subscribe
D. Handle

36. IMPLEMENTATION

37. IMPLEMENTATION
WARNING
ASYNC MESSAGING
!=
ASYNC/AWAIT

38. STEP 1:
COMMAND ISSUING/SENDING

39. IMPLEMENTATION
COMMAND ISSUING - (WEB)APP LAYER
▸ IssueAsync implemented with Bus.Send<T>

40. IMPLEMENTATION
COMMAND SENDING / BOOTSTRAPPING MASSTRANSIT
▸ bus is a Single Instance / AppDomain

41. IMPLEMENTATION
MASSTRANSIT / BUS.SEND<T>
▸ Send(er)/Receive(r) Pattern

42. IMPLEMENTATION
MASSTRANSIT / BUS.SEND<T>
▸ Send(er)/Receive(r) Pattern

43. IMPLEMENTATION
COMMAND INTEGRITY 1/2

44. IMPLEMENTATION
COMMAND INTEGRITY 1/2

45. IMPLEMENTATION
COMMAND INTEGRITY 2/2

46. IMPLEMENTATION
COMMAND INTEGRITY 2/2

47. IMPLEMENTATION
COMMAND ISSUING/SENDING SUMMARY
▸ Use Domain Structures in Command
▸ Command Integrity
▸ ICommandIssuer abstraction
▸ MassTransit implementation
▸ Converts Commands to Messages (TbC)
▸ Encapsulate in an App Service
▸ Validation

48. STEP 2:
COMMAND RECEIVING/
HANDLING

49. IMPLEMENTATION
COMMAND RECEIVING / BOOTSTRAPPING MASSTRANSIT

50. IMPLEMENTATION
COMMAND RECEIVING / BOOTSTRAPPING MASSTRANSIT

51. IMPLEMENTATION
COMMAND RECEIVING / BOOTSTRAPPING MASSTRANSIT

52. IMPLEMENTATION
COMMAND RECEIVING / BOOTSTRAPPING MASSTRANSIT

53. IMPLEMENTATION
COMMAND RECEIVING / BOOTSTRAPPING MASSTRANSIT

54. IMPLEMENTATION
MASSTRANSIT / CONSUMER<T>

55. IMPLEMENTATION
MASSTRANSIT / CONSUMER<T>

56. IMPLEMENTATION
MASSTRANSIT RETRING FLOW (SIMPLIFIED)
1. message deserialised
2. identity consumer from message type
A. if consumer identified
‣ run handler in try/catch
‣ on exception check retry policy
‣ either retry on ignore
B. if consumer is NOT identified (TbC)

57. IMPLEMENTATION
COMMAND HANDLING / EVENT DISPATCHING - APP LAYER

58. IMPLEMENTATION
COMMAND HANDLING / EVENT DISPATCHING - APP LAYER

59. IMPLEMENTATION
AGGREGATE EVENTS
▸ A/Rs should inherit from BaseAggregateRoot
▸ exposes Events collection

60. IMPLEMENTATION
AGGREGATE EVENTS
▸ A/Rs should inherit from BaseAggregateRoot
▸ exposes Events collection

61. IMPLEMENTATION
AGGREGATE EVENTS
▸ A/Rs should inherit from BaseAggregateRoot
▸ exposes Events collection

62. IMPLEMENTATION
EVENT DISPATCHER 1/2
▸ wires SYNChronously (in-process) command with event handlers
▸ is the “Emitter”
▸ wired handlers can be
▸ an actual one (e.g. read model update)
▸ forwarder (event->message + publish) to the bus
▸ both
▸ even for the same event
▸ infra free implementation

63. IMPLEMENTATION
EVENT DISPATCHER 2/2
*
* * * * *
*
* *
* * *
* *
* *
* *
* * *
* * * * *
* * * * * *
* *
* *
* * *
* * *
* *
*
IN-PROCESS
FORWARD
FORWARD

64. STEP 3:
EVENT EMITTING/PUBLISHING

65. IMPLEMENTATION
EVENT EMITTING/PUBLISHING
▸ as with Commands, Events should use domain structures
▸ IEventEmitter abstraction similar to ICommandIssuer
▸ MassTransit implementation with Bus.Publish<T>
▸ Converts Events to Messages (TbC)

66. IMPLEMENTATION
MASSTRANSIT / BUS.PUBLISH<T>
▸ EndPoint agnostic
▸ Publisher(er)/Subscribe(r) Pattern

67. STEP 4:
EVENT SUBSCRIBING/
HANDLING

68. IMPLEMENTATION
EVENT SUBSCRIBING / BOOTSTRAPPING MASSTRANSIT
▸ bootstrapping identical to command receiving
▸ need a Consumer<T>
▸ translates the message to event
▸ instantiates and invokes event handler

69. IMPLEMENTATION
MASSTRANSIT / CONSUMER<T> (AGAIN)

70. IMPLEMENTATION
MASSTRANSIT / CONSUMER<T> (AGAIN)

71. IMPLEMENTATION
EVENT HANDLING

72. IMPLEMENTATION
EVENT HANDLING

73. IMPLEMENTATION
EVENT HANDLING

74. BEST PRACTICES

75. BEST PRACTICES
MESSAGE DESIGN
▸ messages are POCOS. (period)
▸ do NOT reuse domain structures
▸ SERIOUS danger of message loss
▸ type repetition and mapping are unavoidable
▸ Automapper FTW !
▸ messages might change (aka message versioning)
▸ subscribe to interfaces
▸ ensure correct version or serialiser assembly deployed
▸ JSON .NET

76. BEST PRACTICES
SYSTEM ARCHITECTURE
▸ apply selectively not globally
▸ always forward (i.e. publish) the events
▸ bounded context interface
▸ rely on the retrying facilities
▸ idempotency
▸ fence “expensive” services
▸ perhaps even with sync send/receive (e.g. Routing Service)

77. BEST PRACTICES
SCALING OPTIONS
▸ scale up: can invoke Message Handlers in parallel using a
ThreadPool = Round Robin
▸ bus.SetConcurrencyLimit
▸ scale out: “competing consumers”
▸ same EndPointID
▸ probable concurrency issues
▸ aggregate versioning !

78. BEST PRACTICES
HOUSE-KEEPING
▸ descriptive End Point IDs mostly for subscribers/receivers
▸ web_SERVER1_sender
▸ cmd_rec_worker
▸ evt_sub_booking_reporting
▸ evt_sub_shipping
▸ monitor transport

79. BEST PRACTICES
COMPETING CONSUMERS
1 1
1
1 #
1
#

80. INDICATIONS

81. INDICATIONS
▸ command/event handlers that are:
▸ resource intensive/blocking
▸ involve multiple aggregate instances
▸ e.g. reroute all cargos unloading @ Hong Kong
▸ error prone
▸ involve external service calls
▸ e.g. email multiple recipients
▸ multiple bounded contexts

82. INDICATIONS
▸ separate physical read datastores
▸ high load of commands/events
▸ non-human/UI command issuers

83. CHALLENGES

84. CHALLENGES
EVENTUAL CONSISTENCY
▸ heavily affects UI/UX
▸ contradicts user mentality
▸ need to notify on command handling error
▸ makes UI implementation more complex

85. CHALLENGES
IMPLEMENTATION-RELATED
▸ a LOT of type repetition
▸ more physical processes
▸ changes MIGHT need orchestration
▸ concurrency issues
▸ transport = additional infra

86. RESOURCES

87. RESOURCES
READINGS
▸ CQRS Journey by Microsoft Patterns & Practises
cqrsjourney.github.io
▸ CQRS by Edument
cqrs.nu
▸ Clarified CQRS by Udi Dahan
udidahan.com/2009/12/09/clarified-cqrs
▸ MassTransit Documentation
docs.masstransit-project.com

88. RESOURCES
SAMPLES
▸ Cargo Tracker
▸ github.com/citerus/dddsample-core
(Java) by Citerus AB
▸ github.com/SzymonPobiega/DDDSample.Net
(.ΝΕΤ) by Szymon Pobiega
▸ CQRS, The example - Mark Nijhof
leanpub.com/cqrs

89. RESOURCES
SANDBOX SERVICES
▸ RabbitMQ: CloudAMQP
www.cloudamqp.com
▸ MongoDB: Atlas
www.mongodb.com/cloud
▸ PostgreSQL, SQL Server etc: AWS Free Tier
aws.amazon.com/free/

90. Q&A