Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Konrad 'ktoso' Malawski
GeeCON 2014 @ Kraków, PL
Konrad `@ktosopl` Malawski
streams
reactive stream processing
with
Konrad `@ktosopl` Malawski
Akka Team
Reactive Streams TCK
sbt-jmh
…
hAkker @
Konrad `@ktosopl` Malawski
typesafe.com
geecon.org
Java.pl / KrakowScala.pl
sckrk.com / meetup.com/Paper-Cup @ London
GDGK...
Agenda
Agenda
• Reactive Streams
• Background and Specification
• Protocol details
• Akka Streams
• Concepts and goals
• Building ...
Streams
Streams
Streams
“You cannot enter the same river twice”
~ Heraclitus
http://en.wikiquote.org/wiki/Heraclitus
Streams
Real Time Stream Processing
When you attach “late” to a Publisher,
you may miss initial elements – it’s a river of...
Reactive Streams
Reactive Streams
Stream processing
Reactive Streams
Back-pressured
Stream processing
Reactive Streams
Back-pressured
Asynchronous
Stream processing
Reactive Streams
Back-pressured
Asynchronous
Stream processing
Standardised (!)
Reactive Streams: Goals
1. Back-pressured Asynchronous Stream processing
2. Standard implemented by many libraries
Reactive Streams - Specification & TCK
http://reactive-streams.org
Reactive Streams - Who?
http://reactive-streams.org
Kaazing Corp.
rxJava @ Netflix,
reactor @ Pivotal (SpringSource),
vert....
Reactive Streams - Inter-op
http://reactive-streams.org
We want to make different implementations
co-operate with each oth...
Reactive Streams - Inter-op
http://reactive-streams.org
The different implementations “talk to each other”
using the React...
Reactive Streams - Inter-op
http://reactive-streams.org
The Reactive Streams SPI is NOT meant to be user-api.
You should u...
package com.rolandkuhn.rsinterop
import ratpack.rx.RxRatpack
import ratpack.test.embed.EmbeddedApp
import ratpack.handling...
EmbeddedApp.fromHandler(new Handler {
override def handle(ctx: Context): Unit = {
// RxJava Observable
val intObs = Observ...
What is back-pressure?
Back-pressure? Example Without
Publisher[T] Subscriber[T]
Back-pressure? Example Without
Fast Publisher Slow Subscriber
Back-pressure?
“Why would I need that!?”
Back-pressure? Push + NACK model
Back-pressure? Push + NACK model
Subscriber usually has some kind of buffer.
Back-pressure? Push + NACK model
Back-pressure? Push + NACK model
Back-pressure? Push + NACK model
What if the buffer overflows?
Back-pressure? Push + NACK model (a)
Use bounded buffer,
drop messages + require re-sending
Back-pressure? Push + NACK model (a)
Kernel does this!
Routers do this!
(TCP)
Use bounded buffer,
drop messages + require ...
Back-pressure? Push + NACK model (b)
Increase buffer size…
Well, while you have memory available!
Back-pressure? Push + NACK model (b)
Back-pressure?
NACKing is NOT enough.
Negative ACKnowledgement
Back-pressure? Example NACKing
Buffer overflow is imminent!
Back-pressure? Example NACKing
Telling the Publisher to slow down / stop sending…
Back-pressure? Example NACKing
NACK did not make it in time,
because M was in-flight!
Back-pressure?
speed(publisher) < speed(subscriber)
Back-pressure? Fast Subscriber, No Problem
No problem!
Back-pressure?
Reactive-Streams
=
“Dynamic Push/Pull”
Just push – not safe when Slow Subscriber
Just pull – too slow when Fast Subscriber
Back-pressure? RS: Dynamic Push/Pull
Solution:
Dynamic adjustment
Back-pressure? RS: Dynamic Push/Pull
Just push – not safe when Slow Subscriber
Just pull – to...
Back-pressure? RS: Dynamic Push/Pull
Slow Subscriber sees it’s buffer can take 3 elements.
Publisher will never blow up it...
Back-pressure? RS: Dynamic Push/Pull
Fast Publisher will send at-most 3 elements.
This is pull-based-backpressure.
Back-pressure? RS: Dynamic Push/Pull
Fast Subscriber can issue more Request(n),
before more data arrives!
Back-pressure? RS: Dynamic Push/Pull
Fast Subscriber can issue more Request(n),
before more data arrives.
Publisher can ac...
Back-pressure? RS: Accumulate demand
Publisher accumulates total demand per subscriber.
Back-pressure? RS: Accumulate demand
Total demand of elements is safe to publish.
Subscriber’s buffer will not overflow.
Back-pressure? RS: Requesting “a lot”
Fast Subscriber can issue arbitrary large requests,
including “gimme all you got” (L...
Back-pressure? RS: Dynamic Push/Pull
MAX
speed
Back-pressure? RS: Dynamic Push/Pull
Easy
MAX
speed
Back-pressure? RS: Dynamic Push/Pull
Easy
MAX
speed
Pipelining in Reactive Streams
Pipelining in Reactive Streams
A Publisher may be able
to pre-generate some data…
Pipelining in Reactive Streams
Pipelining in Reactive Streams
Request(5)
Pipelining in Reactive Streams
Pipelining in Reactive Streams
Pipelining in Reactive Streams
Pipelining in Reactive Streams
Pipelining in Reactive Streams
Pipelining in Reactive Streams
…and since signalling happens asynchronously…
Pipelining in Reactive Streams
pending demand pending demand
reserve buffer space
Pipelining in Reactive Streams
signal demand
buffer space for
incoming elements
Pipelining in Reactive Streams
The goal here is to never wait, unless back-pressured.
Reactive Streams SPI
Reactive Streams SPI
public interface Publisher<T> {
public void subscribe(Subscriber<? super T> s);
}
Reactive Streams SPI
public interface Publisher<T> {
public void subscribe(Subscriber<? super T> s);
}
gives a public inte...
Reactive Streams SPI
public interface Subscriber<T> {
public void onSubscribe(Subscription s);
public void onNext(T t);
pu...
How does fit in here?
Akka
Akka has multiple modules:
akka-actor: actors (concurrency abstraction)
akka-remote: remote actors
akka-cluster: clus...
Akka
Akka is a high-performance concurrency
library for Scala and Java.
At it’s core it focuses on the Actor Model:
An Actor can only:
• Send and receive messages
• Create Actors
• Change it’s behaviour
Akka
Akka is a high-performance con...
class Player extends Actor {
def receive = {
case NextTurn => sender() ! decideOnMove()
}
def decideOnMove(): Move = ???
}...
Akka
Akka has multiple modules:
akka-actor: actors (concurrency abstraction)
akka-camel: integration
akka-remote: remote a...
streams
Akka Streams – design decisions
Superior:
• reusability
• expansibility
• performance
• bound buffer space
• Java and Scal...
Akka Streams – bounded buffer space
Why reasoning about buffer space is a big deal:
Akka Streams – Linear Flow
Akka Streams – Linear Flow
Akka Streams – Linear Flow
Akka Streams – Linear Flow
Akka Streams – Linear Flow
Flow[Double].map(_.toInt). [...]
No Source attached yet.
“Pipe ready to work with Doubles”.
Akka Streams – Linear Flow
implicit val sys = ActorSystem("tokyo-sys")
An ActorSystem is the world in which Actors live in...
Akka Streams – Linear Flow
implicit val sys = ActorSystem("tokyo-sys")
implicit val mat = FlowMaterializer()
Contains logi...
Akka Streams – Linear Flow
implicit val sys = ActorSystem("tokyo-sys")
implicit val mat = FlowMaterializer()
A materialise...
Akka Streams – Linear Flow
implicit val sys = ActorSystem("tokyo-sys")
implicit val mat = FlowMaterializer()
You can config...
Akka Streams – Linear Flow
implicit val sys = ActorSystem("tokyo-sys")
implicit val mat = FlowMaterializer()
val foreachSi...
Akka Streams – Linear Flow
implicit val sys = ActorSystem("tokyo-sys")
implicit val mat = FlowMaterializer()
val foreachSi...
Akka Streams – Linear Flow
implicit val sys = ActorSystem("tokyo-sys")
implicit val mat = FlowMaterializer()
// sugar for ...
Akka Streams – Linear Flow
val mf = Flow[Int].
map(_ * 2).
runWith(Sink.foreach(println))
// is missing a Source,
// can N...
Akka Streams – Linear Flow
val f = Flow[Int].
map(_ * 2).
runWith(Sink.foreach(i => println(s"i = $i”))).
// needs Source ...
Akka Streams – Linear Flow
val f = Flow[Int].
map(_ * 2).
runWith(Sink.foreach(i => println(s"i = $i”))).
// needs Source ...
Akka Streams – Linear Flow
val f = Flow[Int].
map(_ * 2).
runWith(Sink.foreach(i => println(s"i = $i”))).
// needs Source ...
Akka Streams – Linear Flow
val f = Flow[Int].
map(_ * 2).
runWith(Sink.foreach(i => println(s"i = $i”))).
// needs Source ...
Akka Streams – Linear Flow
val f = Flow[Int].
map(_ * 2).
runWith(Sink.foreach(i => println(s"i = $i”))).
// needs Source ...
Akka Streams – Linear Flow
Flow[Int].
map(_.toString).
runWith(Source(1 to 10), Sink.ignore)
Connects Source and Sink, the...
Akka Streams – Flows are reusable
f.withSource(IterableSource(1 to 10)).run()
f.withSource(IterableSource(1 to 100)).run()...
Akka Streams <-> Actors – Advanced
val subscriber = ActorSubscriber(
system.actorOf(Props[SubStreamParent], ”parent”))
Sou...
Akka Streams <-> Actors – Advanced
Each “group” is a stream too! It’s a “Stream of Streams”.
val subscriber = ActorSubscri...
Akka Streams <-> Actors – Advanced
groupBy(_.last).
GroupBy groups “11” to group “1”, “12” to group “2” etc.
Akka Streams <-> Actors – Advanced
groupBy(_.last).
It offers (groupKey, subStreamSource) to Subscriber
Source
Akka Streams <-> Actors – Advanced
groupBy(_.last).
It can then start children, to handle the sub-flows!
Source
Akka Streams <-> Actors – Advanced
groupBy(_.last).
For example, one child for each group.
Source
Akka Streams <-> Actors – Advanced
val subscriber = ActorSubscriber(
system.actorOf(Props[SubStreamParent], ”parent”))
Sou...
Akka Streams – FlowGraph
FlowGraph
Akka Streams – FlowGraph
Linear Flows
or
non-akka pipelines
Could be another RS implementation!
Akka Streams – GraphFlow
Fan-out elements
and
Fan-in elements
Akka Streams – GraphFlow
// first define some pipeline pieces
val f1 = Flow[Input].map(_.toIntermediate)
val f2 = Flow[Int...
Akka Streams – GraphFlow
Akka Streams – GraphFlow
FlowGraph { implicit b =>
import FlowGraphImplicits._
val bcast = Broadcast[Intermediate]
val mer...
Akka Streams – GraphFlow
FlowGraph { implicit b =>
import FlowGraphImplicits._
val bcast = Broadcast[Intermediate]
val mer...
Akka Streams – GraphFlow
val g = FlowGraph {}
FlowGraph is immutable and safe to share and re-use!
streams
in action
There’s more to explore!
Topics we did explore today:
• asynchronous non-blocking back-pressure
• complex graph processing...
There’s more to explore!
Topics we didn’t explore today:
• explicit buffering, and overflow strategies
• integrating with A...
There’s more to explore!
Future plans:
• API stabilisation and documentation (1.0 soon)
• Improve testability & TestKit
• ...
Links
• http://akka.io
• http://reactive-streams.org

• https://groups.google.com/group/akka-user

• Tim Harper’s awesome ...
©Typesafe 2015 – All Rights Reserved
Reactive
Application
Development!
SBT in Action! Akka in Action!
Play for Scala! Play...
©Typesafe 2015 – All Rights Reserved
Prochain SlideShare
Chargement dans…5
×

Reactive Stream Processing with Akka Streams

13 095 vues

Publié le

Reactive Stream Processing with Akka Streams

  1. 1. Konrad 'ktoso' Malawski GeeCON 2014 @ Kraków, PL Konrad `@ktosopl` Malawski streams reactive stream processing with
  2. 2. Konrad `@ktosopl` Malawski Akka Team Reactive Streams TCK sbt-jmh … hAkker @
  3. 3. Konrad `@ktosopl` Malawski typesafe.com geecon.org Java.pl / KrakowScala.pl sckrk.com / meetup.com/Paper-Cup @ London GDGKrakow.pl meetup.com/Lambda-Lounge-Krakow hAkker @
  4. 4. Agenda
  5. 5. Agenda • Reactive Streams • Background and Specification • Protocol details • Akka Streams • Concepts and goals • Building Blocks • Akka Streams in Action • Q & A
  6. 6. Streams
  7. 7. Streams
  8. 8. Streams “You cannot enter the same river twice” ~ Heraclitus http://en.wikiquote.org/wiki/Heraclitus
  9. 9. Streams Real Time Stream Processing When you attach “late” to a Publisher, you may miss initial elements – it’s a river of data. http://en.wikiquote.org/wiki/Heraclitus
  10. 10. Reactive Streams
  11. 11. Reactive Streams Stream processing
  12. 12. Reactive Streams Back-pressured Stream processing
  13. 13. Reactive Streams Back-pressured Asynchronous Stream processing
  14. 14. Reactive Streams Back-pressured Asynchronous Stream processing Standardised (!)
  15. 15. Reactive Streams: Goals 1. Back-pressured Asynchronous Stream processing 2. Standard implemented by many libraries
  16. 16. Reactive Streams - Specification & TCK http://reactive-streams.org
  17. 17. Reactive Streams - Who? http://reactive-streams.org Kaazing Corp. rxJava @ Netflix, reactor @ Pivotal (SpringSource), vert.x @ Red Hat, Twitter, akka-streams @ Typesafe, spray @ Spray.io, Oracle, java (?) – Doug Lea - SUNY Oswego …
  18. 18. Reactive Streams - Inter-op http://reactive-streams.org We want to make different implementations co-operate with each other.
  19. 19. Reactive Streams - Inter-op http://reactive-streams.org The different implementations “talk to each other” using the Reactive Streams protocol.
  20. 20. Reactive Streams - Inter-op http://reactive-streams.org The Reactive Streams SPI is NOT meant to be user-api. You should use one of the implementing libraries.
  21. 21. package com.rolandkuhn.rsinterop import ratpack.rx.RxRatpack import ratpack.test.embed.EmbeddedApp import ratpack.handling.Handler import ratpack.handling.Context import rx.Observable import scala.collection.JavaConverters._ import akka.stream.scaladsl.Flow import akka.stream.scaladsl.Source import rx.RxReactiveStreams import akka.stream.scaladsl.Sink import akka.actor.ActorSystem import akka.stream.FlowMaterializer import ratpack.http.ResponseChunks import java.util.function.Consumer import ratpack.test.http.TestHttpClient import reactor.rx.Streams object ScalaMain extends App { val system = ActorSystem("InteropTest") implicit val mat = FlowMaterializer()(system) Reactive Streams - Inter-op
  22. 22. EmbeddedApp.fromHandler(new Handler { override def handle(ctx: Context): Unit = { // RxJava Observable val intObs = Observable.from((1 to 10).asJava) // Reactive Streams Publisher val intPub = RxReactiveStreams.toPublisher(intObs) // Akka Streams Source val stringSource = Source(intPub).map(_.toString) // Reactive Streams Publisher val stringPub = stringSource.runWith(Sink.fanoutPublisher(1, 1)) // Reactor Stream val linesStream = Streams.create(stringPub).map[String](new reactor.function.Function[String, String] { override def apply(in: String) = in + "n" }) // and now render the HTTP response (RatPack) ctx.render(ResponseChunks.stringChunks(linesStream)) } }).test(new Consumer[TestHttpClient] { override def accept(client: TestHttpClient): Unit = { val text = client.getText() println(text) system.shutdown() } }) } Reactive Streams - Inter-op
  23. 23. What is back-pressure?
  24. 24. Back-pressure? Example Without Publisher[T] Subscriber[T]
  25. 25. Back-pressure? Example Without Fast Publisher Slow Subscriber
  26. 26. Back-pressure? “Why would I need that!?”
  27. 27. Back-pressure? Push + NACK model
  28. 28. Back-pressure? Push + NACK model Subscriber usually has some kind of buffer.
  29. 29. Back-pressure? Push + NACK model
  30. 30. Back-pressure? Push + NACK model
  31. 31. Back-pressure? Push + NACK model What if the buffer overflows?
  32. 32. Back-pressure? Push + NACK model (a) Use bounded buffer, drop messages + require re-sending
  33. 33. Back-pressure? Push + NACK model (a) Kernel does this! Routers do this! (TCP) Use bounded buffer, drop messages + require re-sending
  34. 34. Back-pressure? Push + NACK model (b) Increase buffer size… Well, while you have memory available!
  35. 35. Back-pressure? Push + NACK model (b)
  36. 36. Back-pressure? NACKing is NOT enough.
  37. 37. Negative ACKnowledgement
  38. 38. Back-pressure? Example NACKing Buffer overflow is imminent!
  39. 39. Back-pressure? Example NACKing Telling the Publisher to slow down / stop sending…
  40. 40. Back-pressure? Example NACKing NACK did not make it in time, because M was in-flight!
  41. 41. Back-pressure? speed(publisher) < speed(subscriber)
  42. 42. Back-pressure? Fast Subscriber, No Problem No problem!
  43. 43. Back-pressure? Reactive-Streams = “Dynamic Push/Pull”
  44. 44. Just push – not safe when Slow Subscriber Just pull – too slow when Fast Subscriber Back-pressure? RS: Dynamic Push/Pull
  45. 45. Solution: Dynamic adjustment Back-pressure? RS: Dynamic Push/Pull Just push – not safe when Slow Subscriber Just pull – too slow when Fast Subscriber
  46. 46. Back-pressure? RS: Dynamic Push/Pull Slow Subscriber sees it’s buffer can take 3 elements. Publisher will never blow up it’s buffer.
  47. 47. Back-pressure? RS: Dynamic Push/Pull Fast Publisher will send at-most 3 elements. This is pull-based-backpressure.
  48. 48. Back-pressure? RS: Dynamic Push/Pull Fast Subscriber can issue more Request(n), before more data arrives!
  49. 49. Back-pressure? RS: Dynamic Push/Pull Fast Subscriber can issue more Request(n), before more data arrives. Publisher can accumulate demand.
  50. 50. Back-pressure? RS: Accumulate demand Publisher accumulates total demand per subscriber.
  51. 51. Back-pressure? RS: Accumulate demand Total demand of elements is safe to publish. Subscriber’s buffer will not overflow.
  52. 52. Back-pressure? RS: Requesting “a lot” Fast Subscriber can issue arbitrary large requests, including “gimme all you got” (Long.MaxValue)
  53. 53. Back-pressure? RS: Dynamic Push/Pull MAX speed
  54. 54. Back-pressure? RS: Dynamic Push/Pull Easy MAX speed
  55. 55. Back-pressure? RS: Dynamic Push/Pull Easy MAX speed
  56. 56. Pipelining in Reactive Streams
  57. 57. Pipelining in Reactive Streams A Publisher may be able to pre-generate some data…
  58. 58. Pipelining in Reactive Streams
  59. 59. Pipelining in Reactive Streams Request(5)
  60. 60. Pipelining in Reactive Streams
  61. 61. Pipelining in Reactive Streams
  62. 62. Pipelining in Reactive Streams
  63. 63. Pipelining in Reactive Streams
  64. 64. Pipelining in Reactive Streams
  65. 65. Pipelining in Reactive Streams …and since signalling happens asynchronously…
  66. 66. Pipelining in Reactive Streams pending demand pending demand reserve buffer space
  67. 67. Pipelining in Reactive Streams signal demand buffer space for incoming elements
  68. 68. Pipelining in Reactive Streams The goal here is to never wait, unless back-pressured.
  69. 69. Reactive Streams SPI
  70. 70. Reactive Streams SPI public interface Publisher<T> { public void subscribe(Subscriber<? super T> s); }
  71. 71. Reactive Streams SPI public interface Publisher<T> { public void subscribe(Subscriber<? super T> s); } gives a public interface Subscription { public void request(long n); public void cancel(); } A
  72. 72. Reactive Streams SPI public interface Subscriber<T> { public void onSubscribe(Subscription s); public void onNext(T t); public void onError(Throwable t); public void onComplete(); } public interface Publisher<T> { public void subscribe(Subscriber<? super T> s); } gives a to a public interface Subscription { public void request(long n); public void cancel(); }
  73. 73. How does fit in here?
  74. 74. Akka Akka has multiple modules: akka-actor: actors (concurrency abstraction) akka-remote: remote actors akka-cluster: clustering akka-persistence: CQRS / Event Sourcing akka-camel: integration akka-streams: stream processing …
  75. 75. Akka Akka is a high-performance concurrency library for Scala and Java. At it’s core it focuses on the Actor Model:
  76. 76. An Actor can only: • Send and receive messages • Create Actors • Change it’s behaviour Akka Akka is a high-performance concurrency library for Scala and Java. At it’s core it focuses on the Actor Model:
  77. 77. class Player extends Actor { def receive = { case NextTurn => sender() ! decideOnMove() } def decideOnMove(): Move = ??? } Akka
  78. 78. Akka Akka has multiple modules: akka-actor: actors (concurrency abstraction) akka-camel: integration akka-remote: remote actors akka-cluster: clustering akka-persistence: CQRS / Event Sourcing akka-streams: stream processing …
  79. 79. streams
  80. 80. Akka Streams – design decisions Superior: • reusability • expansibility • performance • bound buffer space • Java and Scala APIs
  81. 81. Akka Streams – bounded buffer space Why reasoning about buffer space is a big deal:
  82. 82. Akka Streams – Linear Flow
  83. 83. Akka Streams – Linear Flow
  84. 84. Akka Streams – Linear Flow
  85. 85. Akka Streams – Linear Flow
  86. 86. Akka Streams – Linear Flow Flow[Double].map(_.toInt). [...] No Source attached yet. “Pipe ready to work with Doubles”.
  87. 87. Akka Streams – Linear Flow implicit val sys = ActorSystem("tokyo-sys") An ActorSystem is the world in which Actors live in. AkkaStreams uses Actors, so it needs ActorSystem.
  88. 88. Akka Streams – Linear Flow implicit val sys = ActorSystem("tokyo-sys") implicit val mat = FlowMaterializer() Contains logic on HOW to materialise the stream.
  89. 89. Akka Streams – Linear Flow implicit val sys = ActorSystem("tokyo-sys") implicit val mat = FlowMaterializer() A materialiser chooses HOW to materialise a Stream. The Flow’s AST is fully “lifted”. The Materialiser can choose to materialise the Flow in any way it sees fit. Our implementation uses Actors. But you could easily plug in an SparkMaterializer!
  90. 90. Akka Streams – Linear Flow implicit val sys = ActorSystem("tokyo-sys") implicit val mat = FlowMaterializer() You can configure it’s buffer sizes etc.
  91. 91. Akka Streams – Linear Flow implicit val sys = ActorSystem("tokyo-sys") implicit val mat = FlowMaterializer() val foreachSink = Sink.foreach[Int](println) val mf = Source(1 to 3).runWith(foreachSink)
  92. 92. Akka Streams – Linear Flow implicit val sys = ActorSystem("tokyo-sys") implicit val mat = FlowMaterializer() val foreachSink = Sink.foreach[Int](println) val mf = Source(1 to 3).runWith(foreachSink)(mat) Uses the implicit FlowMaterializer
  93. 93. Akka Streams – Linear Flow implicit val sys = ActorSystem("tokyo-sys") implicit val mat = FlowMaterializer() // sugar for runWith Source(1 to 3).foreach(println)
  94. 94. Akka Streams – Linear Flow val mf = Flow[Int]. map(_ * 2). runWith(Sink.foreach(println)) // is missing a Source, // can NOT run == won’t compile!
  95. 95. Akka Streams – Linear Flow val f = Flow[Int]. map(_ * 2). runWith(Sink.foreach(i => println(s"i = $i”))). // needs Source to run!
  96. 96. Akka Streams – Linear Flow val f = Flow[Int]. map(_ * 2). runWith(Sink.foreach(i => println(s"i = $i”))). // needs Source to run!
  97. 97. Akka Streams – Linear Flow val f = Flow[Int]. map(_ * 2). runWith(Sink.foreach(i => println(s"i = $i”))). // needs Source to run!
  98. 98. Akka Streams – Linear Flow val f = Flow[Int]. map(_ * 2). runWith(Sink.foreach(i => println(s"i = $i”))). // needs Source to run! f.connect(Source(1 to 10)).run()
  99. 99. Akka Streams – Linear Flow val f = Flow[Int]. map(_ * 2). runWith(Sink.foreach(i => println(s"i = $i”))). // needs Source to run! f.connect(Source(1 to 10)).run() With a Source attached… it can run()
  100. 100. Akka Streams – Linear Flow Flow[Int]. map(_.toString). runWith(Source(1 to 10), Sink.ignore) Connects Source and Sink, then runs
  101. 101. Akka Streams – Flows are reusable f.withSource(IterableSource(1 to 10)).run() f.withSource(IterableSource(1 to 100)).run() f.withSource(IterableSource(1 to 1000)).run()
  102. 102. Akka Streams <-> Actors – Advanced val subscriber = ActorSubscriber( system.actorOf(Props[SubStreamParent], ”parent”)) Source(1 to 100). map(_.toString). filter(_.length == 2). drop(2). groupBy(_.last). runWith(subscriber)
  103. 103. Akka Streams <-> Actors – Advanced Each “group” is a stream too! It’s a “Stream of Streams”. val subscriber = ActorSubscriber( system.actorOf(Props[SubStreamParent], ”parent”)) Source(1 to 100). map(_.toString). filter(_.length == 2). drop(2). groupBy(_.last). runWith(subscriber)
  104. 104. Akka Streams <-> Actors – Advanced groupBy(_.last). GroupBy groups “11” to group “1”, “12” to group “2” etc.
  105. 105. Akka Streams <-> Actors – Advanced groupBy(_.last). It offers (groupKey, subStreamSource) to Subscriber Source
  106. 106. Akka Streams <-> Actors – Advanced groupBy(_.last). It can then start children, to handle the sub-flows! Source
  107. 107. Akka Streams <-> Actors – Advanced groupBy(_.last). For example, one child for each group. Source
  108. 108. Akka Streams <-> Actors – Advanced val subscriber = ActorSubscriber( system.actorOf(Props[SubStreamParent], ”parent”)) Source(1 to 100). map(_.toString). filter(_.length == 2). drop(2). groupBy(_.last). runWith(subscriber) The Actor, will consume SubStream offers.
  109. 109. Akka Streams – FlowGraph FlowGraph
  110. 110. Akka Streams – FlowGraph Linear Flows or non-akka pipelines Could be another RS implementation!
  111. 111. Akka Streams – GraphFlow Fan-out elements and Fan-in elements
  112. 112. Akka Streams – GraphFlow // first define some pipeline pieces val f1 = Flow[Input].map(_.toIntermediate) val f2 = Flow[Intermediate].map(_.enrich) val f3 = Flow[Enriched].filter(_.isImportant) val f4 = Flow[Intermediate].mapFuture(_.enrichAsync) // then add input and output placeholders val in = SubscriberSource[Input] val out = PublisherSink[Enriched]
  113. 113. Akka Streams – GraphFlow
  114. 114. Akka Streams – GraphFlow FlowGraph { implicit b => import FlowGraphImplicits._ val bcast = Broadcast[Intermediate] val merge = Merge[Enriched] in ~> f1 ~> bcast ~> f2 ~> merge bcast ~> f4 ~> merge ~> f3 ~> out }
  115. 115. Akka Streams – GraphFlow FlowGraph { implicit b => import FlowGraphImplicits._ val bcast = Broadcast[Intermediate] val merge = Merge[Enriched] in ~> f1 ~> bcast ~> f2 ~> merge bcast ~> f4 ~> merge ~> f3 ~> out }
  116. 116. Akka Streams – GraphFlow val g = FlowGraph {} FlowGraph is immutable and safe to share and re-use!
  117. 117. streams in action
  118. 118. There’s more to explore! Topics we did explore today: • asynchronous non-blocking back-pressure • complex graph processing pipelines • streams powered TCP server / client • a sneak peek into custom elements
  119. 119. There’s more to explore! Topics we didn’t explore today: • explicit buffering, and overflow strategies • integrating with Akka Actors • time-based operators (takeWhile, dropWhile, timer transforms) • plenty additional combinators and junctions • implementing custom processing stages and junctions
  120. 120. There’s more to explore! Future plans: • API stabilisation and documentation (1.0 soon) • Improve testability & TestKit • Performance tuning of Streams & HTTP • Provide more Sinks / Sources and operations • Visualising flow graphs • great experiment by Tim Harper
 https://github.com/timcharper/reactive-viz • Distributing computation graphs (?)
  121. 121. Links • http://akka.io • http://reactive-streams.org
 • https://groups.google.com/group/akka-user
 • Tim Harper’s awesome complex pipeline example + visualisation
 https://github.com/timcharper/reactive-viz
 • 1.0-M2 Documentation (not complete)
 http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0-M2/scala.html • Complete JavaDSL for all operations
 https://github.com/akka/akka/pulls?q=is%3Apr+javadsl
  122. 122. ©Typesafe 2015 – All Rights Reserved Reactive Application Development! SBT in Action! Akka in Action! Play for Scala! Play for Java! Atomic Scala! Check out eBooks at Typesafe!! Visit http://typesafe.com/resources/e-books to view all!! Check out eBooks at Typesafe Visit http://typesafe.com/resources/e-books to view all  
  123. 123. ©Typesafe 2015 – All Rights Reserved

×