Understand the different approaches to integrate fast data and streams based frameworks into your legacy applications and learn about the advantages, disadvantages, caveats, and bumps in the road.

1. Reactive Integrations - Caveats and
bumps in the road explained

2. @myfear

3. Why is everybody talking about
cloud and microservices
and what the **** is streaming?

4. High Infrastructure
Cost
11%
Awful Downtime
9%
Meeting Demand
21%
Release Frquency
20%
Developer Velocity
39%
Biggest Problems in Software Development
• People
• Methodology
• Code
• Systems

5. http://www.internetlivestats.com/internet-users/
More Users

6. More Data
• Rather than acting on data at rest,
modern software increasingly
operates on data in near real-time.
• Shortened time-frames for putting
changes into production
• New business models evolve from
existing ones
• New questions need to be answered
by existing applications

7. Can’t really be served with these
CENTRALISED
MAINFRAME
DECENTRALIZED
APPSERVERS

8. DISTRIBUTED
CLOUD
That’s why we have
MICROSERVICES

9. That includes
CENTRALISED
MAINFRAME
DECENTRALIZED
APPSERVERS
DISTRIBUTED
CLOUD

10. MORE
USERS
MORE
DATA
MORE
SYSTEMS
+ + =
MORE
CONNECTIONS
And ultimately means

11. Scaling the most common programming task…
INPUT
Transform Output

12. …to work in a data intensive, world.
highly asynchronous,

13. But isn’t that _just_ integration?

14. Consumer Producer
Make
request
Wait
for
response
Consumer Producer
Make
request
Continue
Work
Do
something
with
response
Synchronous vs. Asynchronous

15. But there are already APIs for that?!
• InputStream and OutputStream in the JDK
• NIO Channel’s
• Servlet 3.1 ReadListener and WriteListener
extensions
• JDBC ResultSet
• JSR 356 @OnMessage annotations,
• Message Driven Beans and JMS,
• CDI events using @Observes
• Java collection Stream and Iterator based APIs
• JDK9 Flow API.

16. Service A
Let’s add integration to it
Transform
Service C
Possibly
asynchronous
boundaries

17. Service A
Let’s add integration to it
Transform
Service C
10 msg/s 1 msg/s

18. Service A
Let’s add integration to it
Transform
Service C
10 msg/s 1 msg/s
java.lang.OutOfMemoryError

19. What is needed is Flow Control

20. Searching for a stream library
that can handle backpressure

21. Stream all the things!
map,flatMap,filter,groupBy

22. Reactive Streams is an initiative to provide a
standard for asynchronous stream processing
with non-blocking back pressure. This
encompasses efforts aimed at runtime
environments (JVM and JavaScript) as well as
network protocols.
http://www.reactive-streams.org
Reactive Streams to the rescue
https://developer.lightbend.com/blog/2017-08-18-introduction-to-reactive-streams-for-java-developers/index.html

23. Towards reusable shapes for integration
Source<Integer>
Integer
Flow<Integer, String> Sink<String>
Integer String String
Source<Integer,NotUsed>source =
Source.range(0,20000000);
Flow<Integer,String,NotUsed>flow =
Flow.fromFunction(
(Integern) ->n.toString()
);
Sink<String,CompletionStage<Done>>sink =
Sink.foreach(str->
System.out.println(str)
);

24. What are the real usecases?

25. 1) Modernization / Distributed Systems
DB Application
Reactive
Microservices
Mainframe
Cloud
Services
Storage
Security
Search
… Application

26. Spark
DB
2) Moving ETL/Batch style applications…
DB Sqoop
Resource
Managers
Persistence
Batch
YARN

27. Events
Message Broker Akka Streams
DB
Microservices
2) …to Streaming Architectures.
Persistence
Low Latency
Microservices

28. Wanna learn more about
Streaming Architectures?
http://bit.ly/fast-data-architecture

29. DB Application
Reactive
Microservices
Mainframe
Cloud
Services
Storage
Security
Search
…
Application
• REST
• JMS
• SSE
• JDBC
• WebSocket
• ?????

30. “It depends”

31. DB Application
Reactive
Microservices
JDBC
WebSocket
SSE
REST

32. The important parts are “streaming” and
“backpressure”

33. Meet Alpakka!

34. Reactive Enterprise Integration - Alpakka Project
https://developer.lightbend.com/docs/alpakka/current/
•AMQP Connector
•Apache Geode connector
•AWS DynamoDB Connector
•AWS Kinesis Connector
•AWS Lambda Connector
•AWS S3 Connector
•AWS SNS Connector
•AWS SQS Connector
•Azure Storage Queue Connector
•Cassandra Connector
•Elasticsearch Connector
•File Connectors
•FTP Connector
•Google Cloud Pub/Sub
•HBase connector
•IronMq Connector
•JMS Connector
•MongoDB Connector
•MQTT Connector
•Server-sent Events (SSE) Connector
•Slick (JDBC) Connector
•Spring Web
•Unix Domain Socket Connector
•File IO
•Azure
•Camel
•Eventuate
•FS2
•HTTP Client
•MongoDB
•Kafka
•TCP

35. SlickH2
Alpakka
Slick
Akka
Streams
Akka HTTP
An example .. Finally :-D
Alpakka

36. <dependency>
<groupId>com.lightbend.akka</groupId>
<artifactId>akka-stream-alpakka-slick_${scala.version}</artifactId>
<version>${akka.alpakka.version}</version>
</dependency>
<dependency>
<groupId>com.typesafe.akka</groupId>
<artifactId>akka-stream_${scala.version}</artifactId>
<version>${akka.version}</version>
</dependency>
<dependency>
<groupId>com.typesafe.akka</groupId>
<artifactId>akka-http_${scala.version}</artifactId>
<version>${akka.http.version}</version>
</dependency>
<scala.version>2.12</scala.version>
<akka.version>2.5.9</akka.version>
<akka.http.version>10.0.10</akka.http.version>
<akka.alpakka.version>0.16</akka.alpakka.version>

37. mvn compile exec:java
http://localhost:8080/ http://localhost:8080/more http://localhost:8080/

38. https://github.com/myfear/alpakka-jdbc

39. //Sample Source
Source<String,NotUsed>source = Source.from(Arrays.asList("1-2-3","2-3","3-4"));
CompletionStage<List<Integer>> ret =
source.map(s->Arrays.asList(s.split("-")))
.mapConcat(f->f)
//Sub-streams logic
.map(s->Integer.valueOf(s))
.runWith(Sink.seq(),materializer);
//Verify results
List<Integer>list = ret.toCompletableFuture().get();
assertlist.equals(Arrays.asList(1,2,3,2,3,3,4));
Integration Pattern Example

40. https://developer.lightbend.com/blog/2018-02-06-reactive-streams-ee4j/index.html
What can Reactive Streams offer EE4J?

41. May the Source[T, _] be with you!

42. Next Steps! Learn more Akka and Alpakka
Project Site:
http://www.lightbend.com/akka
GitHub Repo:
https://github.com/akka
Documentation:
https://developer.lightbend.com/docs/alpakka/current/
Example:
https://github.com/myfear/alpakka-jdbc

43. Written for architects and developers that must
quickly gain a fundamental understanding of
microservice-based architectures, this free O’Reilly
report explores the journey from SOA to
microservices, discusses approaches to dismantling
your monolith, and reviews the key tenets of a
Reactive microservice:
• Isolate all the Things
• Act Autonomously
• Do One Thing, and Do It Well
• Own Your State, Exclusively
• Embrace Asynchronous Message-Passing
• Stay Mobile, but Addressable
• Collaborate as Systems to Solve Problems
http://bit.ly/ReactiveMicroservice

44. The detailed example in this report is based on
Lagom, a new framework that helps you follow the
requirements for building distributed, reactive
systems.
• Get an overview of the Reactive Programming
model and basic requirements for developing
reactive microservices
• Learn how to create base services, expose
endpoints, and then connect them with a
simple, web-based user interface
• Understand how to deal with persistence, state,
and clients
• Use integration technologies to start a
successful migration away from legacy systems
http://bit.ly/DevelopReactiveMicroservice

45. http://bit.ly/SustainableEnterprise
• Understand the challenges of starting a greenfield
development vs tearing apart an existing brownfield
application into services
• Examine your business domain to see if microservices
would be a good fit
• Explore best practices for automation, high availability,
data separation, and performance
• Align your development teams around business
capabilities and responsibilities
• Inspect design patterns such as aggregator, proxy,
pipeline, or shared resources to model service
interactions