Cloud computing, reactive systems, microservices: distributed programming has become the norm. But while the shift to loosely coupled message-based systems has manifest benefits in terms of resilience and elasticity, our tools for ensuring correct behavior has not grown at the same pace. Statically typed languages like Java and Scala allow us to exclude large classes of programming errors before the first test is run. Unfortunately, these guarantees are limited to the local behavior within a single process, the compiler cannot tell us that we are sending the wrong JSON structure to a given web service. Therefore distribution comes at the cost of having to write large test suites, with timing-dependent non-determinism. In this presentation we take a first peek at ways out of this dilemma. The principles are demonstrated on the simplest distributed system: Actors. We show how parameterized ActorRefs à la Akka Typed together with effect tracking similar to HLists can help us define what an Actor can and cannot do during its lifetime—and have the compiler yell at us when we do it wrong.

1. Taming Distribution:
Formal Protocols
for Akka Typed
Dr. Roland Kuhn
@rolandkuhn — CTO of Actyx

2. Distribution
=
Concurrency + Partial Failure

3. Distribution
=
Concurrency + Partial Failure

4. Distribution
=
Concurrency + Partial Failure

5. Actors model distribution.

6. Concurrency implies Nondeterminism.
Distribution implies more Nondeterminism.

7. Concurrency implies Nondeterminism.
Distribution implies more Nondeterminism.

8. Causality restricts Nondeterminism.

9. Some causality comes naturally.

10. Akka Typed Receptionist API
trait ServiceKey[T]
sealed trait Command
final case class Register[T](key: ServiceKey[T], address: ActorRef[T],
replyTo: ActorRef[Registered[T]]) extends Command
final case class Registered[T](key: ServiceKey[T], address: ActorRef[T])
final case class Find[T](key: ServiceKey[T], replyTo: ActorRef[Listing[T]]) extends Command
final case class Listing[T](key: ServiceKey[T], addresses: Set[ActorRef[T]])

11. … with Unregister support
trait ServiceKey[T]
sealed trait Command
final case class Register[T](key: ServiceKey[T], address: ActorRef[T],
replyTo: ActorRef[Registered[T]]) extends Command
final case class Registered[T](key: ServiceKey[T], address: ActorRef[T], handle: ActorRef[Unregister])
final case class Unregister(replyTo: ActorRef[Unregistered])
final case class Unregistered()
final case class Find[T](key: ServiceKey[T], replyTo: ActorRef[Listing[T]]) extends Command
final case class Listing[T](key: ServiceKey[T], addresses: Set[ActorRef[T]])

12. For everything that is not fixed by causality
coordination is needed.

13. Static knowledge avoids coordination.

14. Cluster Receptionist

15. Cluster Receptionist
• use FQCN of service keys as known identifier

16. Cluster Receptionist
• use FQCN of service keys as known identifier
• local resolution establishes static type-safety

17. Cluster Receptionist
• use FQCN of service keys as known identifier
• local resolution establishes static type-safety
• CRDT map from keys to sets of ActorRefs

18. Natural causality is not enough!

19. Example: payment with audit

20. Messages for a payment system
case object AuditService extends ServiceKey[LogActivity]
case class LogActivity(who: ActorRef[Nothing], what: String,
id: Long, replyTo: ActorRef[ActivityLogged])
case class ActivityLogged(who: ActorRef[Nothing], id: Long)

21. Messages for a payment system
case object AuditService extends ServiceKey[LogActivity]
case class LogActivity(who: ActorRef[Nothing], what: String,
id: Long, replyTo: ActorRef[ActivityLogged])
case class ActivityLogged(who: ActorRef[Nothing], id: Long)
sealed trait PaymentService
case class Authorize(payer: URI, amount: BigDecimal, id: UUID, replyTo: ActorRef[PaymentResult])
extends PaymentService
case class Capture(id: UUID, amount: BigDecimal, replyTo: ActorRef[PaymentResult])
extends PaymentService
case class Void(id: UUID, replyTo: ActorRef[PaymentResult]) extends PaymentService
case class Refund(id: UUID, replyTo: ActorRef[PaymentResult]) extends PaymentService

22. Messages for a payment system
case object AuditService extends ServiceKey[LogActivity]
case class LogActivity(who: ActorRef[Nothing], what: String,
id: Long, replyTo: ActorRef[ActivityLogged])
case class ActivityLogged(who: ActorRef[Nothing], id: Long)
sealed trait PaymentService
case class Authorize(payer: URI, amount: BigDecimal, id: UUID, replyTo: ActorRef[PaymentResult])
extends PaymentService
case class Capture(id: UUID, amount: BigDecimal, replyTo: ActorRef[PaymentResult])
extends PaymentService
case class Void(id: UUID, replyTo: ActorRef[PaymentResult]) extends PaymentService
case class Refund(id: UUID, replyTo: ActorRef[PaymentResult]) extends PaymentService
sealed trait PaymentResult
case class PaymentSuccess(id: UUID) extends PaymentResult
case class PaymentRejected(id: UUID, reason: String) extends PaymentResult
case class IdUnkwown(id: UUID) extends PaymentResult

23. Akka Typed crash course
case class Greet(whom: String)
class Greeter extends akka.actor.Actor {
def receive = {
case Greet(whom) => println(s"Hello $whom!")
}
}

24. Akka Typed crash course
case class Greet(whom: String)
class Greeter extends akka.actor.Actor {
def receive = {
case Greet(whom) => println(s"Hello $whom!")
}
}
object Greeter {
import akka.typed.scaladsl.Actor
val behavior =
Actor.immutable[Greet] { (ctx, greet) =>
println(s"Hello ${greet.whom}!")
Actor.same
}
}

25. First actor: do the audit
sealed trait Msg
private case object AuditDone extends Msg
private case object PaymentDone extends Msg
private def doAudit(audit: ActorRef[LogActivity], who: ActorRef[AuditDone.type], msg: String) =
Actor.deferred[ActivityLogged] { ctx =>
val id = Random.nextLong()
audit ! LogActivity(who, msg, id, ctx.self)
ctx.schedule(3.seconds, ctx.self, ActivityLogged(null, 0L))
Actor.immutable { (ctx, msg) =>
if (msg.who == null) throw new TimeoutException
else if (msg.id != id) throw new IllegalStateException
else {
who ! AuditDone
Actor.stopped
}
}
}

26. Second actor: do the payment
private def doPayment(from: URI, amount: BigDecimal, payments: ActorRef[PaymentService],
replyTo: ActorRef[PaymentDone.type]) =
Actor.deferred[PaymentResult] { ctx =>
val uuid = UUID.randomUUID()
payments ! Authorize(from, amount, uuid, ctx.self)
ctx.schedule(3.seconds, ctx.self, IdUnkwown(null))
Actor.immutable {
case (ctx, PaymentSuccess(`uuid`)) =>
payments ! Capture(uuid, amount, ctx.self)
Actor.immutable {
case (ctx, PaymentSuccess(`uuid`)) =>
replyTo ! PaymentDone
Actor.stopped
}
// otherwise die with MatchError
}
}

27. Third actor: orchestration of the process
def getMoney[R](from: URI, amount: BigDecimal,
payments: ActorRef[PaymentService], audit: ActorRef[LogActivity],
replyTo: ActorRef[R], msg: R) =
Actor.deferred[Msg] { ctx =>
ctx.watch(ctx.spawn(doAudit(audit, ctx.self, "starting payment"), "preAudit"))
Actor.immutable[Msg] {
case (ctx, AuditDone) =>
ctx.watch(ctx.spawn(doPayment(from, amount, payments, ctx.self), "payment"))
Actor.immutable[Msg] {
case (ctx, PaymentDone) =>
ctx.watch(ctx.spawn(doAudit(audit, ctx.self, "payment finished"), "postAudit"))
Actor.immutable[Msg] {
case (ctx, AuditDone) =>
replyTo ! msg
Actor.stopped
}
} onSignal terminateUponChildFailure
} onSignal terminateUponChildFailure
}

28. code can employ knowledge in wrong order
or
existing knowledge is not used at all

29. What if we prescribe effects and their order?

30. Which steps shall be done?
• send audit log, get confirmation for that
• send Authorize request, get confirmation
• send Capture request, get confirmation
• send audit log, get confirmation

31. Akka Typed Session: protocol definition
object GetMoneyProtocol extends Protocol {
type Session = //
Send[LogActivity] :: // preAudit
Read[ActivityLogged] :: //
Choice[(Halt :: _0) :+: _0 :+: CNil] :: // possibly terminate
Send[Authorize] :: // do payment
Read[PaymentResult] :: //
Choice[(Halt :: _0) :+: _0 :+: CNil] :: // possibly terminate
Send[Capture] :: //
Read[PaymentResult] :: //
Choice[(Halt :: _0) :+: _0 :+: CNil] :: // possibly terminate
Send[LogActivity] :: // postAudit
Read[ActivityLogged] :: //
Choice[(Halt :: _0) :+: _0 :+: CNil] :: // possibly terminate
_0
}

32. First process: do the audit
private def doAudit(audit: ActorRef[LogActivity], who: ActorRef[Nothing], msg: String) =
OpDSL[ActivityLogged] { implicit opDSL =>
val id = Random.nextLong()
for {
self <- opProcessSelf
_ <- opSend(audit, LogActivity(who, msg, id, self))
ActivityLogged(`who`, `id`) <- opRead
} yield Done
}.withTimeout(3.seconds)
/*
* the return type is
* Process[ActivityLogged, Done,
* Send[LogActivity] :: Read[ActivityLogged] ::
* Choice[(Halt :: _0) :+: _0 :+: CNil] :: _0]
*/

33. Second process: do the payment
private def doPayment(from: URI, amount: BigDecimal, payments: ActorRef[PaymentService]) =
OpDSL[PaymentResult] { implicit opDSL =>
val uuid = UUID.randomUUID()
for {
self <- opProcessSelf
_ <- opSend(payments, Authorize(from, amount, uuid, self))
PaymentSuccess(`uuid`) <- opRead
_ <- opSend(payments, Capture(uuid, amount, self))
PaymentSuccess(`uuid`) <- opRead
} yield Done
}.withTimeout(3.seconds)
/*
* the return type is
* Process[PaymentResult, Done,
* Send[Authorize] :: Read[PaymentResult] ::
* Choice[(Halt :: _0) :+: _0 :+: CNil] ::
* Send[Capture] :: Read[PaymentResult] ::
* Choice[(Halt :: _0) :+: _0 :+: CNil] :: _0]
*/

34. Third process: orchestrate and verify
// this useful process is provided by the Session DSL and talks to the Receptionist
def getService[T](key: ServiceKey[T]): Operation[Listing[T], ActorRef[T], _0]
def getMoney[R](from: URI, amount: BigDecimal,
payments: ActorRef[PaymentService],
replyTo: ActorRef[R], msg: R) =
OpDSL[Nothing] { implicit opDSL =>
for {
self <- opProcessSelf
audit <- opCall(getService(AuditService) .named("getAuditService"))
_ <- opCall(doAudit(audit, self, "starting payment").named("preAudit"))
_ <- opCall(doPayment(from, amount, payments) .named("payment"))
_ <- opCall(doAudit(audit, self, "payment finished").named("postAudit"))
} yield replyTo ! msg
}

35. Third process: orchestrate and verify
// this useful process is provided by the Session DSL and talks to the Receptionist
def getService[T](key: ServiceKey[T]): Operation[Listing[T], ActorRef[T], _0]
def getMoney[R](from: URI, amount: BigDecimal,
payments: ActorRef[PaymentService],
replyTo: ActorRef[R], msg: R) =
OpDSL[Nothing] { implicit opDSL =>
for {
self <- opProcessSelf
audit <- opCall(getService(AuditService) .named("getAuditService"))
_ <- opCall(doAudit(audit, self, "starting payment").named("preAudit"))
_ <- opCall(doPayment(from, amount, payments) .named("payment"))
_ <- opCall(doAudit(audit, self, "payment finished").named("postAudit"))
} yield replyTo ! msg
}
// compile-time verification (TODO: should be a macro)
private def verify = E.vetExternalProtocol(GetMoneyProtocol, getMoney(???, ???, ???, ???, ???))

36. Conclusion:
There is a lot on the table,
get involved!
https://github.com/rkuhn/akka-typed-session