NServiceBus - building a distributed system based on a messaging infrastructure

1. NServiceBus: building a distributed
system based on a messaging
infrastructure

2. Mauro Servienti
CTO @ Mastreeno, LTD
mauro.servienti@mastreeno.com
@mauroservienti
//milestone.topics.it
//github.com/mauroservienti
NServiceBus trainer/support
RavenDB trainer
Microsoft MVP – Visual C#

3. Resources
• Slides on SlideShare:
//slideshare.net/mauroservienti
• Samples on GitHub:
//github.com/mauroservienti
– Repository: Conferences
– Branches:
• Full-Stack-Sample
• 2014/Italian-Developers-Meet-up

4. TENETS
Articles of faith…

5. None of the following is true
• Network is reliable;
• Latency is near to zero or irrelevant;
• Bandwidth is unlimited;
• Network is secure;
• Topology doesn’t change;
• Transport cost is irrelevant;
• Network is homogeneous;

6. DEFINITIONS
Let’s get in touch…

7. Consistency
The rate of agreement of observers looking at a
system at a given point in time.
The more the observers agree on what they see
the more the system is consistent.

8. Coupling
The rate of dependency among parts of a
system.
The more changing a portion of the system
impacts on other portions of the system the
more the system is coupled.

9. Temporal Coupling
It’s a special form of coupling.
The more non-availability of a portion of the
system impacts on other parts the more the
system is temporally coupled.

10. Scalability
the ability of a system to handle a growing
amount of work in a capable manner
Scalability is generally difficult to define
and in any particular case it is necessary to
define the specific requirements for
scalability

11. The more we scale the
more we cannot rely on
consistency

12. “ACD/C”
Scaling can be achieved understanding that we
need to choose and accept consequences of our
decisions, our pillars should be:
- Asynchronous;
- Cached;
- Distributed;
- And not Consistent;

13. CONSINSTENCY?
Are you kidding me :-)

14. A strange world :-)
• A new order comes in;
• The whole company is informed that a new order
we’ll be processed and we need to:
– Understand if items are in stock;
– Understand if we need to produce/buy something;
• At the same time production is trying to
understand how to schedule the new order but is
waiting for the warehouse that is currently used
by the sales department to understand if the
order can be shipped within the next week;

15. DEADLOCK

16. The real world…
• The obvious and only consequence of trying
to scale a monolith is the collapse of the
entire system;
• The real world:
– Does not know at all what transactions are
(especially distributed);
– Has a really low, if not null, coupling among parts;
– Has no temporal coupling at all;

17. Transaction boundaries
• We cannot any more rely on transactions to
guarantee consistency, e.g.:
1. Update the shopping chart;
2. Checkout;
3. Create the order;
4. Create the shipment request at FedEx;
• “Simply” 1, 2, 3 and 4 can live in different systems on
different machines with different databases;
– And given our tenets we now have a problem
– And a solution... :-)

18. EVENTUAL
CONSISTENCY

19. The rate of the agreement
• Will be low or really low;
• Every communication must bring with itself its
version (or timestamp) in order to be able to
sort stuff;
• Parts of the system are now free to move
independently:
– They can evolve due to the low coupling;
– Be available or not, depending on their needs,
because there is no temporal coupling;

20. MESSAGES
We have async and distributed parts…but…how they talk to each other?

21. asynchronous
We cannot rely on RPC calls
the other part is not guaranteed to be there
when we need it.

22. asynchronous
QueueSender Receiver
Now
Some time in the future

23. distributed
We need an atomic piece of information
we cannot rely on ordering
we cannot rely on receiving the information at
the same place (is distributed);

24. distributed
QueueSender Receiver
C
Receiver
Receiver
B
A
B
A
Broken!
C

25. non-coupled
We need the message as small as possible
The more the exchanged vocabulary is large the
more coupling we have
Changing the vocabulary is hard, think twice
about it

26. NServiceBus
Please welcome:

27. Concepts
• Message
– An atomic piece of information that has a semantical
meaning in the business;
• Component
– Something that can handle a message;
• Service
– A set of components grouped by context;
• Endpoint
– A set of services grouped by:
• SLA(s);
• Infrastructure concerns;
• Etc..;

28. Concepts #2
• Command: A message that semantically
identifies something to be done (imperative):
– "CreateNewUser";
• Event: A message that semantically identifies
something happened and immutable (past):
– "NewUserCreated";
• Subscription: The notion that an endpoint is
interested in an event;

29. DEMO

30. Transport
• Transport(s): The technology used to connect
systems and transport the message:
– MSMQ
– RabbitMQ
– SQL Server
– Azure ServiceBus & Azure Queues
• Serialization: the way messages are"serialized" in
order to be transported on the choosen
transport;
– it is transparent to the transport;

31. Advanced Concepts
• Saga: An orchestrator for a long running
workflow, with the ability to store the saga
state across requests and handling
concurrency;
• Timeout: The way a Saga can take
autonomous decisions;
• Retries: First level and Second level retry
engine to handle transient failures;
• Error & Audit: error and auditing management

32. SCALE OUT & HIGH AVAILABILITY
In an eventual consistent world

33. Mail & Mail Servers
When we send an email message:
• Our relation with the mail server is consistent?
Yes;
• Cross-servers relation is consistent? Yes;
• Relationship between the last server and the
recipient is consistent? Yes;

34. The entire system is consistent? No
But we have some guarantees:
• Every single hop/node/BC is consistent;
• If something along the way fails we will have,
with the same logic, an information back that
our request is failed or succeeded;
• Do we need distributed transactions? No :-)
• The message is fully enough to guarantee
consistency, in the long run.

35. DEMO

36. Publish/Subscribe
• Request/Response is generally considered an
anti-pattern;
• Events are the easiest way to drive the world:
– SomethingHappened;
• DoSomethingToMoveOn;
• Lots of possible listener and lots of possible
publishers:
– CorrelationID

37. DEMO

38. QUESTIONS?
We are all set :-)