Developers are quickly moving to having Apache Kafka and events at the heart of their architecture. But how do you make sure your applications are resilient to the fluctuating load that comes with a never-ending stream of events? The Reactive Manifesto provides a good starting point for these kinds of problems. In this session explore how Kafka and reactive application architecture can be combined to better handle our modern event-streaming needs. We will explain why reactive applications are a great fit for Kafka and show an example of how to write a reactive producer and consumer.

1. Reacting to
an Event-Driven world
© 2020 IBM Corporation
Kate Stanley and Grace Jansen

2. IBM Event Streams
Let’s get some coffee…
© 2020 IBM Corporation

3. IBM Event Streams
Barista Example:
© 2020 IBM Corporation
Coffee Shop
Barista
Coffee Lovers
HTTP
HTTP
https://github.com/cescoffier/quarkus-coffeeshop-demo

4. IBM Event Streams
Barista Example:
© 2020 IBM Corporation
Coffee Shop
Coffee Shop
Barista
Barista
Board
Coffee Lovers
Coffee Lovers
HTTP
Orders,
Queue
HTTP
HTTP
https://github.com/cescoffier/quarkus-coffeeshop-demo
Event
Backbone

5. IBM Event Streams© 2020 IBM Corporation
Q: Is your coffee shop non-blocking and
highly responsive?

6. IBM Event Streams© 2020 IBM Corporation
Q: Is your microservice system non-
blocking and highly responsive?

7. IBM Event Streams© 2020 IBM Corporation
Q: Is your microservice system non-
blocking and highly responsive?
A: Yes I’m using Kafka!

8. IBM Event Streams© 2020 IBM Corporation
Q: Is your microservice system non-
blocking and highly responsive?
A: Yes I’m using Kafka!

9. IBM Event Streams
Building reactive
systems
© 2020 IBM Corporation

10. IBM Event Streams
Reactive Manifesto
© 2020 IBM Corporation

11. IBM Event Streams
Message-Driven
Reactive Manifesto
© 2020 IBM Corporation

12. IBM Event Streams
Events
“A signal emitted by a component upon
reaching a given state.”
© 2020 IBM Corporation
A message can contain an encoded event in
its payload.
Messages
“An item of data sent to a specific location.”

13. IBM Event Streams© 2020 IBM Corporation
Apache Kafka is an open source, distributed
streaming platform
Publish and subscribe to streams of
records
Store records in durable way
Process streams of records as they
occur

14. IBM Event Streams
Barista Example:
© 2020 IBM Corporation
Coffee Shop
Barista
Board
Coffee Lovers
Orders,
Queue
HTTP
https://github.com/cescoffier/quarkus-coffeeshop-demo
Event
Backbone

15. IBM Event Streams
Barista Example:
© 2020 IBM Corporation
Coffee Shop
Barista
Board
Coffee Lovers
Orders,
Queue
HTTP
https://github.com/cescoffier/quarkus-coffeeshop-demo
Event
Backbone

16. IBM Event Streams
Barista Example:
© 2020 IBM Corporation
Coffee Shop
Barista
Board
Coffee Lovers
Orders,
Queue
HTTP
https://github.com/cescoffier/quarkus-coffeeshop-demo
Event
Backbone

17. IBM Event Streams
Elastic
Message-Driven
Reactive Manifesto
© 2020 IBM Corporation

18. IBM Event Streams
Barista Example:
© 2020 IBM Corporation
Coffee Shop
Barista
Board
Coffee Lovers
Orders,
Queue
HTTP
https://github.com/cescoffier/quarkus-coffeeshop-demo
Event
Backbone

19. IBM Event Streams
Barista Example:
© 2020 IBM Corporation
Coffee Shop
Barista
Board
Coffee Lovers
Orders,
Queue
HTTP
https://github.com/cescoffier/quarkus-coffeeshop-demo
Event
Backbone

20. IBM Event Streams
Barista Example:
© 2020 IBM Corporation
Coffee Shop
Barista
Board
Coffee Lovers
Orders,
Queue
HTTP
https://github.com/cescoffier/quarkus-coffeeshop-demo
Event
Backbone

21. IBM Event Streams
ResilientElastic
Message-Driven
Reactive Manifesto
© 2020 IBM Corporation

22. IBM Event Streams
ResilientElastic
Responsive
Message-Driven
Reactive Manifesto
© 2020 IBM Corporation

23. IBM Event Streams
ResilientElastic
Responsive
Message-Driven
Reactive Manifesto
© 2020 IBM Corporation

24. IBM Event Streams
How do we make a highly responsive app?
Microservice 1
© 2020 IBM Corporation
Microservice 2
Microservice 3

25. IBM Event Streams
How do we make a highly responsive app?
Microservice 1
© 2020 IBM Corporation
Microservice 2
Microservice 3
Reactive
Architecture
Patterns

26. IBM Event Streams
How do we make a highly responsive app?
Microservice 1
© 2020 IBM Corporation
Microservice 2
Microservice 3
Futures
Reactive Streams
Libraries
Futures
Reactive Streams
Libraries
Futures
Reactive Streams
Libraries
Reactive
Programming

27. IBM Event Streams
Integrating reactively
with Kafka
© 2020 IBM Corporation

28. IBM Event Streams
Reactive features of Kafka
© 2020 IBM Corporation
Consumer
offsets
Consumer
groups
Replication
Resiliency Elasticity
Stream history
Partitions
Cruise control

29. IBM Event Streams
Reactive Programming
© 2020 IBM Corporation
A subset of asynchronous programming and a paradigm where the
availability of new information drives the logic forward rather than
having control flow driven by a thread-of-execution.

30. IBM Event Streams© 2020 IBM Corporation

31. IBM Event Streams
send(record)
App code Kafka client
Kafka Producer
© 2020 IBM Corporation
buffer

32. IBM Event Streams
send(record)
App code Kafka client
Kafka Producer
© 2020 IBM Corporation
buffer

33. IBM Event Streams
send(record)
App code Kafka client
Kafka Producer
© 2020 IBM Corporation
buffer
java.util.concurrent.Future

34. IBM Event Streams
send(record)
App code Kafka client
Kafka Producer
© 2020 IBM Corporation
buffer
java.util.concurrent.Future

35. IBM Event Streams
Kafka Consumer
© 2020 IBM Corporation
poll()
App code Kafka client
process()

36. IBM Event Streams
Kafka Consumer
© 2020 IBM Corporation
poll()
App code Kafka client
process()

37. IBM Event Streams
Kafka Consumer
© 2020 IBM Corporation
poll()
App code Kafka client
process()

38. IBM Event Streams
Kafka Consumer
© 2020 IBM Corporation
poll()
App code Kafka client
process()

39. IBM Event Streams
Kafka Consumer
© 2020 IBM Corporation
poll()
App code Kafka client
process()

40. IBM Event Streams
Kafka Consumer
© 2020 IBM Corporation
poll()
App code Kafka client
process()

41. IBM Event Streams
Kafka Consumer
© 2020 IBM Corporation
poll()
App code Kafka client
process()

42. IBM Event Streams
Kafka Consumer
© 2020 IBM Corporation
poll()
App code Kafka client
process()
< max.poll.interval.ms

43. IBM Event Streams
Kafka Consumer
© 2020 IBM Corporation
poll()
App code
process()
process()
process()
process()

44. IBM Event Streams
Reactive Streams is a standard and specification for
stream-oriented libraries for the JVM that..
© 2020 IBM Corporation
…process a potentially unbounded
number of elements in sequence…
…asynchronously passing elements
between components…
…with mandatory non-blocking
backpressure.
App

45. IBM Event Streams
Reactive Streams is a standard and specification for
stream-oriented libraries for the JVM that..
© 2020 IBM Corporation
…process a potentially unbounded
number of elements in sequence…
…asynchronously passing elements
between components…
…with mandatory non-blocking
backpressure.
App
…

46. IBM Event Streams
Reactive Streams is a standard and specification for
stream-oriented libraries for the JVM that..
© 2020 IBM Corporation
…process a potentially unbounded
number of elements in sequence…
…asynchronously passing elements
between components…
…with mandatory non-blocking
backpressure.
App

47. IBM Event Streams
Reactive Streams is a standard and specification for
stream-oriented libraries for the JVM that..
© 2020 IBM Corporation
…process a potentially unbounded
number of elements in sequence…
…asynchronously passing elements
between components…
…with mandatory non-blocking
backpressure.
App

48. IBM Event Streams
Reactive Frameworks for
Kafka
Alpakka Kafka Connector
MicroProfile Reactive Messaging
Vert.x Kafka Client
© 2020 IBM Corporation

49. IBM Event Streams
MicroProfile 3.3 Stack
© 2020 IBM Corporation
Reactive Streams
Operators 1.1
GraphQL
1.0
Standalone Projects
JSON-B 1.0JSON-P 1.1CDI 2.0
Config 1.4
Fault
Tolerance 2.1
JWT
Propagation
1.1
Health
Check 2.2
Metrics 2.3
Open Tracing
1.3
Open API 1.1
JAX-RS 2.1 Rest Client 1.4
Reactive
Messaging 1.0
Context
Propagation
1.0
Reactive
Streams
Not a MicroProfile spec

50. IBM Event Streams
MicroProfile Reactive Messaging API
© 2020 IBM Corporation

51. IBM Event Streams
Alpakka Kafka Connector API
© 2020 IBM Corporation

52. IBM Event Streams
Backpressure in Reactive
Streams
© 2020 IBM Corporation
AppApp

53. IBM Event Streams
Backpressure in Reactive
Streams
© 2020 IBM Corporation
App
Publisher SubscriberProcessor

54. IBM Event Streams
Backpressure in Reactive
Streams
© 2020 IBM Corporation
App
Publisher SubscriberProcessor
Request

55. IBM Event Streams
Backpressure in Reactive
Streams
© 2020 IBM Corporation
App
Publisher SubscriberProcessor
Push-mode

56. IBM Event Streams
Backpressure in Reactive
Streams
© 2020 IBM Corporation
App
Publisher SubscriberProcessor
Push-mode
Poll

57. IBM Event Streams
App
Publisher SubscriberProcessor
Backpressure in Reactive
Streams
© 2020 IBM Corporation
Poll

58. IBM Event Streams
Backpressure in Reactive
Streams
© 2020 IBM Corporation
App
Publisher SubscriberProcessor
Poll
Pause
Backpressure

59. IBM Event Streams
Backpressure in Reactive
Streams
© 2020 IBM Corporation
App
Publisher SubscriberProcessor
Poll
Pause
Pull-mode

60. IBM Event Streams
Backpressure in Reactive
Streams
© 2020 IBM Corporation
App
Publisher SubscriberProcessor
Poll
Pause
Request
Pull-mode

61. IBM Event Streams
Backpressure in Reactive
Streams
© 2020 IBM Corporation
App
Publisher SubscriberProcessor
Pull-mode
Poll
Resume

62. IBM Event Streams© 2020 IBM Corporation
https://www.lightbend.com/blog/alpakka-kafka-flow-control-optimizations
https://issues.apache.org/jira/browse/KAFKA-7548

63. IBM Event Streams
Vert.x Demo App
© 2020 IBM Corporation

64. IBM Event Streams
Eclipse Vert.x
© 2020 IBM Corporation

65. IBM Event Streams
Demo app
© 2020 IBM Corporation
https://github.com/ibm-messaging/kafka-java-vertx-starter
Vert.x app
produce
consume

66. IBM Event Streams
Demo app
© 2020 IBM Corporation
Vert.x app
websocket
produce
consume
https://github.com/ibm-messaging/kafka-java-vertx-starter

67. IBM Event Streams© 2020 IBM Corporation

68. IBM Event Streams© 2020 IBM Corporation
https://github.com/ibm-messaging/kafka-java-vertx-starter

69. IBM Event Streams© 2020 IBM Corporation
ibm.biz/ExperiencesWritingAReactiveKafkaApp

70. IBM Event Streams
Summary
Non-reactive + Kafka != reactive
Consider Kafka configuration for the best
reactive system
The open-source reactive community is on
hand to help!
© 2020 IBM Corporation

71. IBM Event Streams
Reactive Systems Explained
© 2020 IBM Corporation
ibm.biz/ReactiveReport

72. IBM Event Streams
MicroProfile Reactive Messaging
Guide
© 2020 IBM Corporation
ibm.biz/ReactiveMessagingGuide

73. IBM Event Streams
IBM Event Streams
Apache Kafka for the enterprise
© 2020 IBM Corporation
ibm.biz/aboutEventStreams
Powerful Ops Tooling
Award Winning User Experience
Connector Catalog
Support you can Trust
Unrivalled MQ connectivity
Schema Registry
Geo-replication for DR

74. Thank you
Grace Jansen | @gracejansen27
Kate Stanley | @katestanley91
© 2020 IBM Corporation IBM Event Streams: ibm.biz/aboutEventStreams
Reactive resources:
https://www.reactivemanifesto.org
https://ibm.biz/GettingStartedWithReactive
Getting started with Kafka:
https://kafka.apache.org/quickstart
https://strimzi.io
Reactive Kafka libraries
https://vertx.io/docs/vertx-kafka-client/java/
https://github.com/eclipse/microprofile-reactive-
messaging
https://projectreactor.io/docs/kafka
https://github.com/akka/alpakka-kafka