This document discusses how to build a scalable and resilient system to retrieve insurance data from over 400 backends in real-time. It recommends using an actor model approach with message-driven asynchronous communication between components. Key aspects include using a routing engine to distribute requests, implementing a standardized interface for each backend, and incorporating circuit breakers to handle failures without bringing down the whole system. The reactive manifesto principles of being message-driven, elastic, resilient and responsive are important to stay performant under varying workloads.

1. HOW TO STAY
RESPONSIVE WITH 400
BACKENDS
@mthmuldersmaartenm@infosupport.com

5. So, we need something that is...
Mobile First
Scalable
Secure
... but ...
doesn't store insurance data
retrieves that in real-time
from ~ 400 different companies!

8. START AS SIMPLE AS POSSIBLE

9. All problems in computer science can be
solved by another abstraction layer.
-- not David Wheeler (1927 - 2004)

10. JUST MOVE THE PROBLEM AWAY
Main Systems
Routing Engine
Insurer 01 Insurer 02 Insurer 03 .... Insurer nn

11. FIRE UP YOUR IDE & START CODING
Standardized interface for retrieving insurance data.
One endpoint address for each insurance company.
Retrieve data using link code that includes insurer ID.
Shouldn't be too hard!

14. YES, IT BREAKS...

15. NOT RELATED, BUT
ANALYSIS
Blocking network calls
No time-outs
The whole JVM is waiting!
If an insurer is down, it will still be contacted.
This will slow down the system even further.

16. THE REACTIVE MANIFESTO

18. MESSAGE DRIVEN
Reactive Systems rely on asynchronous
message-passing to establish a boundary
between components that ensures loose
coupling, isolation and location
transparency.

19. ELASTIC
The system stays responsive under varying
workload. Reactive Systems can react to
changes in the input rate by increasing or
decreasing the resources allocated to
service these inputs.

20. RESILIENT
The system stays responsive in the face of
failure. Failures are contained within each
component, [...] thereby ensuring that
parts of the system can fail and recover
without compromising the system as a
whole.

21. RESPONSIVE
The system responds in a timely manner if
at all possible. Responsiveness is the
cornerstone of usability and utility [...].
Responsive systems focus on providing
rapid and consistent response times [...].

22. MEET... THE ACTOR MODEL

25. THE CIRCUIT BREAKER

26. GIMME TEH CODEZ!

27. INITIATE RETRIEVAL WITH TIME-OUT
val actor = context.actorOf(
ConsultInsurersActor.props(), s"consult-$berrfnr" )
actor ! RetrieveAllPolicyData(linkingCode)
context.system.scheduler.scheduleOnce(2 seconds) {
self ! RetrievalTimeout(berrfnr, linkingCodes)
}

28. CONSULT ONE INSURER
// Send the request
val request = HttpRequest(POST, Uri(url), Nil, entity)
http ? request
// Receive the response
response.entity.as[ResponseData] match {
case Left(failure) => client ! Failure(failure)
case Right(data) => client ! PolicyDataRetrieved(data)
}

29. CONSULT INSURER (WITH CIRCUIT BREAKER)
val cb = new CircuitBreaker(system.scheduler,
maxFailures = 5,
callTimeout = 5 seconds,
resetTimeout = 1 minute) // or obtain from config
// Send the request
val request = HttpRequest(POST, Uri(url), Nil, entity)
cb.withCircuitBreaker(http ? request) pipeTo self

34. KEY TAKEAWAYS
Actor model enables building resilient solutions
Suitable for large portions of concurrent work
Think outside the box
Use a whiteboard

35. GRACIAS
¿PREGUNTAS?
IMAGE ATTRIBUTIONS
Insurance Policy by .
"Success Kid" a.k.a Sammy Griner by .
Blendtec founder/owner Tom Dickson by .
Reactive Manifesto by .
Actor Model by .
Pictures of Money (Flickr)
Laney Griner (Flickr)
Tim Provost
Wiljag Denekamp
Wiljag Denekamp