Event Driven Architectures

1. Event Driven Architectures
Avinash Ramineni

2. • What is Event Driven Architecture
• How is EDA related to SOA
• Event Capture/Generation
• Event Processing
• Event Delivery Guarantees
• Complex Event Processing
• Events, Internet of Things and Big Data
Agenda

3. • Significant change in the state of an Object
– Bank Transaction
– Students grade gets posted
– New Order gets created
– Customer Profile gets updated
– Change in Heart Beat
• Characteristics of an Event
– Events are broadcast
– Communications are asynchronous
– Events are fine grained
Event

4. • Handling of the Events as they occur and the transfer of
events among the systems in a loosely coupled way
• Enables Situation Awareness and enables sense-and-respond
behavior
Event Driven Architecture (EDA)

5. • Characteristics of EDA
– Producer/Consumers are not aware of each other
– Little or no statefullness
– Distributed
– Loosely Coupled
– Platform and Language Independent
– Reliable
– Fast and Highly Scalable
Event Driven Architecture (EDA)

6. EDA extension of SOA
Computing
with event
objects
REST, URIs and
HTTP
Event Processing
RPC-style SOA
Event-driven
SOA
SOA
Web Architecture
Component
software done
right
Web-oriented
architecture
(WOA)
Non-SOA EDA
and CEP
Simple Web
sites

7. • EDA similar to SOA facilitates Agility by using Modular Design
and Separation of Concerns
• EDA fits well into organizations that have some SOA
infrastructure in place
• EDA adds another dimension / Possibilities to SOA
– Events are generally stateless and do not have much content
• While processing the events that do not have much data in them, the consuming
applications call the services exposed to get the actual data
• Publish – Subscribe Model
– Similar to Asynchronous Messaging Architectures
EDA extension to SOA

8. • Application Generated
– Application generates an event when a business transaction
completes
– Events generated by parsing the access logs/ server logs
– Polling web pages
– capturing the events from the services
• Database Generated
– Database Trigger generated Events
• On insert/update/delete
– During database replication
How to generate Events

9. • The network is reliable
• Latency is zero
• Bandwidth is infinite
• The network is secure
• Topology does not change
• Transport cost is zero
• Network is homogeneous
Fallacies of Distributed Systems

10. • Send an email when a customers profile gets updated
Customer Profile
– Assumptions:
• There is an Application A that provides a updateProfileService
• There is an Application B that provides a sendEmailService
• All the Events are delivered with Once and Only Once SLA and the Events
are processed
Usecase: 1
A
Event
Channel
DB
Update DB
Profile Updated Event
B

11. • The overhead involved in capturing events must be as low as
possible to avoid impairing the performance of a business
transaction.
• Business Transaction and the Event generation needs to be in
the same transaction
– 1,2 needs to be part of a same transaction
• Both 1 and 2 should succeed or both should fail
Considerations while generating Events
A
Event
Channel
DB
1. Update DB
2. Profile Updated Event
B

12. • Successful Update of Profile -- Call a service to publish a
Profile update Event
• Can a database transaction and Web service call be made
atomic?
Usecase: 1 – Web service
try{
1. Update Profile on DB
2. Call web service to publish profile
updated event
} catch(Exception e)
{
rollback();
}
commit;
try{
1. Update Profile on DB
} catch(Exception e)
{
rollback();
}
commit();
2. Call web service to publish profile
updated event
• Can Compensating Transaction work ?

13. • Successful Update of Profile -- Put a message on to Message
Broker
Usecase: 1 – JMS
A Event Channel
DB
1. Update DB
2. Profile Updated Event
(JMS message)
B
• Possible to make 1 and 2 into a single transaction but it
requires – Distributed Transaction.

14. • Follows 2 Phase Commit protocol
• Very Chatty protocol and does a lot of logging to be able to
recover from any failure scenario.
• Too much overhead to 99.9% of the cases to handle less than
0.1% of cases
• Increases the chances for deadlocks on the database.
• Lowers the overall performance of the system.
• Bane of scalability. Grinds the entire system to halt by adding
overhead to each and every transaction.
Distributed/XA Transactions (1)

15. • Availability of the Systems goes down.
• XA Configuration is complicated
• Difficult to test.
• Many People tend to believe that using JTA implementation of
transaction manager will take care of a XA transactions
• Many a time people end up using JTA Manager even while
dealing with single resource.
Distributed/XA Transactions (2)

16. • Successful Update of Profile -- Put a message on to Message
Broker
Usecase: 1 – Local Database
A
Event Channel
DB
1. Update DB
3. Batch process to send
events to Event Channel
B
• 1 and 2 are in same transaction. 3 can be retried multiple
times till they succeed.
2. Store Event

17. • Successful Update of Profile -- Put a message on to Message
Broker
Usecase: 1 – Local JMS
A
Event Channel
DB
1. Update DB
3. Publish to Event
Channel(JMS message)
B
• Possible to make 1 and 2 into a single transaction without a
Distributed Transaction as long as Queue is backed by a same
database
queue
Send Event to Local Queue
2

18. • Successful Update of Profile -- Put a message on to Message
Broker / Call a web service to publish an Event
• Have a Reconciliation Process in place to verify that there is
an event generated for every Business Transaction
– Have a unique Id (may be stored in DB) along with update and use that
Id as Correlation Id with the Events Archive
– If it events don’t tally up, recreate those events
• Use the DB triggers to generate events
– Write to a different local table
– Call a web service to send the event to event channel (have seen
people do that)
Usecase: 1 – Other Ways

19. • Event Delivery Guarantees
– Reliability
• At least Once
– Duplicate events can be delivered
• At most Once
– Some events can be lost
• Once and only Once
– Each is delivered Exactly once
– Order of Delivery
• In Order Guaranteed
• In Order not Guaranteed
Event Delivery

20. • Processing an Event and associated Business Transaction
needs to be in the same transaction
• 3,2 needs to be part of a same transaction
• Both 3 and 2 should succeed or both should fail
Considerations while Processing
Events
Event
Channel
3. Acknowledge Event
Process success
1. Profile Updated Event
B
2. Send Email

21. • On Profile Update Event -- Call a service to send Email and on
success acknowledge process successful
Usecase: 1 – Consumer (1)
onMessage
try {
sendEmail();
jms.commit()
}
catch (Exception e) {
jms.rollback()
}
• What if jms.commit() fails ??

22. • If Jms.commit() fails , message gets delivered again
Usecase: 1 – Consumer (2)
onMessage
try {
if I have not processed this message successfully before {
do some stuff in the database / Idempotency Logic /JMSRelivered flag
jdbc.commit;
}
jms.commit()
}
catch (Exception e) {
jms.rollback()
}

23. • Idempotent Operation
– An idempotent operations means that the result of a successful
performed request is independent of the number of times it is
executed.
• Design consumer to be an Idempotent Consumer
– By Understanding the Business rules it can be achieved relatively easy
• Good Practice to make all the write operations Idempotent !!
• Idempotency logic can be built on Event Ids, Business Ids –
orderId, customerId, etc based on the Business scenario
Work Around for At-Least-Once

24. • In-order delivery of events
– Limit the number of event publisher/consumer instance
for these specific events to 1
• Brings up questions like if one of the event is having an issue – all
the events of that event type can come to a stand still
– AVOID the need for In-order Delivery of Events
• By limiting the content in the Event , we can avoid a lot of cases
where In-order delivery is required
• Using Message Selectors to select and aggregate the events can
solve a few of the cases
What If In-order Delivery is required

25. • Student Information System (SIS) maintain a list of students in
a course roster. The students can be added or dropped from
the course roster. SIS sends an event to Learning Management
System (LMS) when the roster gets updated.
• Lets us assume that SIS is sending the course roster as part of
the event.
• Processing of this Event requires the Roster Change events to
be delivered in order to maintain the data from getting
corrupted.
Usecase: 2

26. Usecase: 2 – Requires In-Order
SIS
Event
Channel
DB
2. Deliver Roster
Change Event
LMS
LMS DB
1. Roster Change Event

27. • Remove the roster data from the event and just pass a roster
Id
• On Delivery of the Event , the LMS system would use the
rosterId from the event, makes a service call to the source
system (SIS) to get the latest Roster data and builds the
classes in LMS
• Even if the events are delivered out of order , the roster data
on LMS will be always be the latest.
• By making the processor on LMS idempotent, we can even
avoid the database update
Usecase: 2 - Solution

28. Usecase: 2
SIS
Event
Channel
DB
2. Deliver Roster
Change Event
LMS
LMS DB
1. Roster Change Event
3. Get the latest roster
for the Id
4. Update the LMS
database

29. • Make Event Publishers responsible for making sure that
events are generated for every Business transaction and are
published to the event channel
• Make Event Consumers responsible for making sure that they
react to the Stimuli from the event
• Dashboards in place to make sure failed Events are detected
and acted upon
• Collects Stats to see if the events need to be made finer or
coarser
• Maintain Event Catalogs and self service capabilities to
subscribe to events
Best Practices

30. Complex Event Processing (CEP)
Traditional BI finds "needles
in haystacks."
Event-driven CEP finds
"needles" in continuously
arriving streams of "hay."
• CEP tap the events happening with in an organization and
provide a situation awareness and enable better and faster
decisions
– Identify complicated events that are inferred from event pattern
detection, event pattern interpretors and so on
– Ex: Monitoring – capture events from ticket system, network load,
performance and send out notification bring up new servers into the
pool

31. • With Big Data hype and with the “capture-it-all“ approach and
IoT , EDA and Event Driven programming will get a huge boost
in the coming years.
• Events and EDA due to their characteristics will be very well
suited for Ubiquitous Computing
• We might start seeing an advent of Complex Event
Applications by being used by various gadgets
– The concepts of EDA and Events will be a perfect Match for gadgets like Basis ,FIT
Events ,Big Data and Internet of Things

32. Industry ExperienceQuestions ?
avinash@clairvoyantsoft.com
Twitter:@avinashramineni