Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Introduction to Akka Streams [Part-II]

1 429 vues

Publié le

Part I - Advanced concepts of Akka Streams.

Publié dans : Logiciels
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Introduction to Akka Streams [Part-II]

  1. 1. By: Harmeet Singh (Taara) Sr. Software Consultant
  2. 2. Agenda 1. Flows 2. Operator Fusion. 3. Graph DSL. 4. Leftover.
  3. 3. Akka Stream: Flows The way to perform transformations to the data as it flows between Source and Sink.
  4. 4. Akka Stream: Flows
  5. 5. Akka Stream: Example object StreamingCopyApp extends App {   val spath = Paths.get("/home/harmeet/akka­stream/knolx.log")   val source: Source[ByteString, Future[IOResult]] =  FileIO.fromPath(spath)   val dpath = Paths.get("/home/harmeet/akka­stream­copy")   val sink: Sink[ByteString, Future[IOResult]] = FileIO.toPath(dpath)   val runnableGraph: RunnableGraph[Future[IOResult]] = source.to(sink)   implicit val system = ActorSystem("akka­stream")   implicit val ec = system.dispatcher   implicit  val materializer = ActorMaterializer()   runnableGraph.run().foreach { result =>     println(s"${result.status}, ${result.count} bytes read")     system.terminate()   } }
  6. 6. Akka Stream: Flow Example val frame: Flow[ByteString, String, NotUsed] =  Framing.delimiter(ByteString("n"), 10240).map(_.decodeString("UTF8")) val parse: Flow[String, Event, NotUsed] = Flow[String].map(Event.parsing) val filter: Flow[Event, Event, NotUsed] = Flow[Event].filter(_.log ==  "DEBUG") val serialize: Flow[Event, ByteString, NotUsed] = Flow[Event].map{ event =>     ByteString(event.toJson.compactPrint) } val composedFlow: Flow[ByteString, ByteString, NotUsed] = frame.via(parse)     .via(filter)     .via(serialize) val runnableGraph: RunnableGraph[Future[IOResult]] =  source.via(composedFlow).toMat(sink) (Keep.right)
  7. 7. Akka Stream: Flow Example val composedFlow: Flow[ByteString, ByteString, NotUsed] =  Framing.delimiter(ByteString("n"), 10240)     .map(_.decodeString("UTF8"))     .map(Event.parsing)     .filter(_.log == "DEBUG")     .map { event =>       ByteString(event.toJson.compactPrint)     } val runnableGraph: RunnableGraph[Future[IOResult]] =  source.via(composedFlow)     .toMat(sink) (Keep.right)
  8. 8. Akka Stream: Operator Fusion Operator Fusion is used to performance optimization to mitigate the cost of passing elements between the different stages. Two main consequences are: 1. Passing elements fused stages faster. 2. Fused stages no longer run in parallel.
  9. 9. Akka Stream: Operator Fusion Q. How can we execute fused stages in parallel??? Ans: 1. By using async method 2. Turn off auto-fusing (akka.stream.materializer.auto­ fusing=off) Example: Source(List(1, 2, 3)).map(_ + 1) .async .map(_ * 2).to(Sink.ignore)
  10. 10. Akka Stream: Operator Fusion
  11. 11. Akka Stream: Graph DSL What if your data flow cannot be modeled as a simple linear process? What if process is better modeled as computation graph instead?
  12. 12. Akka Stream: Graph DSL The Graph DSL is kind of diagram ASCII art – in many cases you could translate a white board diagram of a graph into the DSL. A graph problem is one that typically involves the concept of fanning out and fanning in. ● Fanning out: Single input to multiple output. ● Fanning in: Multiple inputs single output.
  13. 13. Akka Stream: Graph DSL Example val in = Source(1 to 5)          val f1 = Flow[Int].map(_ * 2)    val f2 = Flow[Int].map(_ * 1)
  14. 14. Akka Stream: Graph DSL Example val f3 = Flow[Int].map(_ * 2)         val f4 = Flow[Int].map(_ + 1) val out = Sink.foreach[Int](println)
  15. 15. Akka Stream: Graph DSL Example val bcast = builder.add(Broadcast[Int](2)) val merge = builder.add(Merge[Int](2))
  16. 16. Akka Stream: Graph DSL Example  in ~> f1 ~> bcast ~> f2 ~> merge ~> f4 ~> out
  17. 17. Akka Stream: Graph DSL Example        bcast ~> f3 ~> merge
  18. 18. Akka Stream: Graph DSL Example val graph = RunnableGraph.fromGraph(GraphDSL.create() {     implicit builder: GraphDSL.Builder[NotUsed] =>       import GraphDSL.Implicits._       val in = Source(1 to 5)       val out = Sink.foreach[Int](println)       val f1 = Flow[Int].map(_ * 2)       val f2 = Flow[Int].map(_ * 1)       val f3 = Flow[Int].map(_ * 2)       val f4 = Flow[Int].map(_ + 1)       val bcast = builder.add(Broadcast[Int](2))       val merge = builder.add(Merge[Int](2))       in ~> f1 ~> bcast ~> f2 ~> merge ~> f4 ~> out       bcast ~> f3 ~> merge       ClosedShape   })
  19. 19. Akka Stream: Graph DSL ClosedShape Thegraph itself wasfully self-contained and complete. OR It doesnot haveany open input port and output port like aclosed circuit.
  20. 20. Akka Stream: Partial Graphs Whenever weneed to createareusablecomponent for graph, which wecan useaccording to requirementswith existing or new graph. https://doc.akka.io/docs/akka/2.5/scala/stream/stream-graphs.html#constructing-and-combinin
  21. 21. Leftovers ➔ API’s ➔ Error Handling. ➔ Streaming HTTP ➔ Handling IO Stream ➔ Unit Testing
  22. 22. References: ✔ AkkainAction by Raymond Roestenburg, Rob Bakker, Rob Williams. ✔ MasteringAkkaby Christian Baxter ✔ https://doc.akka.io/api/akka/current/akka/stream/index.html?_ga=2.25006541

×