Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Prochain SlideShare
Chargement dans…5
×

# Scala.io

Presentation of Finagle, Twitter's RPC library, ScalaIO 2014.

• Full Name
Comment goes here.

Are you sure you want to Yes No
• Identifiez-vous pour voir les commentaires

### Scala.io

1. 1. Finagle: Twitter’s RPC Library 24 Oct. 2014 Steve Gury @stevegury
2. 2. > whoami stevegury 2005 2008 2011 2014 time Distributed Systems Massively Multiplayer Online Games Twitter Scala
3. 3. Agenda 1. Motivation 2.Finagle TL;DR 3.Three Key Abstractions 4.Finagle Big Picture
4. 4. Motivation & Context 2006 2010 2014 >2M Tweets / day (January 2009) >65M Tweets / day (July 2010) >500M Tweets / day (August 2013)
5. 5. Agenda 1. Motivation 2.Finagle TL;DR 3.Three Key Abstractions 4.Finagle Big Picture
6. 6. Finagle TL;DR Compose RPC like you compose functions
7. 7. Composing Functions getUserId(name: String): Int getTweets(userId: Int): Tweets getTweets(name: String): Tweets = getUserId ◦ getTweets (name)
8. 8. Agenda 1. Motivation 2.Finagle TL;DR 3.Three Key Abstractions 4.Finagle Big Picture
9. 9. Futures Futures are containers for value Pending Successful Failed Present
10. 10. Futures a val a: Future[Int]
11. 11. Futures a b val a: Future[Int] val b: Future[Int] = a map { x => x + 512 }
12. 12. Futures a b c val a: Future[Int] val b: Future[Int] = a map { x => x + 512 } val c = a map { x => 64 / x }
13. 13. Futures a b c d val a: Future[Int] val b: Future[Int] = a map { x => x + 512 } val c = a map { x => 64 / x } val d = Future.join(b,c)
14. 14. Futures a b c d e val a: Future[Int] val b: Future[Int] = a map { x => x + 512 } val c = a map { x => 64 / x } val d = Future.join(b,c) val e = d map { case (x, y) => x + y }
15. 15. Futures 16 b a c d e val a: Future[Int] val b: Future[Int] = a map { x => x + 512 } val c = a map { x => 64 / x } val d = Future.join(b,c) val e = d map { case (x, y) => x + y }
16. 16. Futures 16 b a c d e val a: Future[Int] val b: Future[Int] = a map { x => x + 512 } val c = a map { x => 64 / x } val d = Future.join(b,c) val e = d map { case (x, y) => x + y }
17. 17. Futures 16 a 528 b c d e val a: Future[Int] val b: Future[Int] = a map { x => x + 512 } val c = a map { x => 64 / x } val d = Future.join(b,c) val e = d map { case (x, y) => x + y }
18. 18. Futures 16 a 528 b c d e val a: Future[Int] val b: Future[Int] = a map { x => x + 512 } val c = a map { x => 64 / x } val d = Future.join(b,c) val e = d map { case (x, y) => x + y }
19. 19. Futures 16 a 528 b 4 c d e val a: Future[Int] val b: Future[Int] = a map { x => x + 512 } val c = a map { x => 64 / x } val d = Future.join(b,c) val e = d map { case (x, y) => x + y }
20. 20. Futures 16 a 528 b 4 c d e val a: Future[Int] val b: Future[Int] = a map { x => x + 512 } val c = a map { x => 64 / x } val d = Future.join(b,c) val e = d map { case (x, y) => x + y }
21. 21. Futures 16 a 528 b 4 c d e val a: Future[Int] val b: Future[Int] = a map { x => x + 512 } val c = a map { x => 64 / x } val d = Future.join(b,c) val e = d map { case (x, y) => x + y }
22. 22. Futures 16 a 528 b 4 (528, 4) c d e val a: Future[Int] val b: Future[Int] = a map { x => x + 512 } val c = a map { x => 64 / x } val d = Future.join(b,c) val e = d map { case (x, y) => x + y }
23. 23. Futures 16 a 528 b 4 (528, 4) c d e val a: Future[Int] val b: Future[Int] = a map { x => x + 512 } val c = a map { x => 64 / x } val d = Future.join(b,c) val e = d map { case (x, y) => x + y }
24. 24. Futures 16 a 528 b 532 4 (528, 4) c d e val a: Future[Int] val b: Future[Int] = a map { x => x + 512 } val c = a map { x => 64 / x } val d = Future.join(b,c) val e = d map { case (x, y) => x + y }
25. 25. Futures 0 b a c d e val a: Future[Int] val b: Future[Int] = a map { x => x + 512 } val c = a map { x => 64 / x } val d = Future.join(b,c) val e = d map { case (x, y) => x + y }
26. 26. Futures 0 a 512 b c d e val a: Future[Int] val b: Future[Int] = a map { x => x + 512 } val c = a map { x => 64 / x } val d = Future.join(b,c) val e = d map { case (x, y) => x + y }
27. 27. Futures 0 a 512 b Ex c d e val a: Future[Int] val b: Future[Int] = a map { x => x + 512 } val c = a map { x => 64 / x } val d = Future.join(b,c) val e = d map { case (x, y) => x + y }
28. 28. Futures 0 a 512 b Ex Ex c d e val a: Future[Int] val b: Future[Int] = a map { x => x + 512 } val c = a map { x => 64 / x } val d = Future.join(b,c) val e = d map { case (x, y) => x + y }
29. 29. Futures 0 a 512 b Ex Ex Ex c d e val a: Future[Int] val b: Future[Int] = a map { x => x + 512 } val c = a map { x => 64 / x } val d = Future.join(b,c) val e = d map { case (x, y) => x + y }
30. 30. Futures Lots of combinators: ! map -> add an edge+node in the graph flatMap -> combine two graphs together handle -> deal with exceptions join -> merge multiple nodes of the graphs select -> select the first node with a value ! …more
31. 31. Programming with Futures == Building a Graph
32. 32. Service Remember the key idea: treat RPC like functions
33. 33. Service Service is a Function from Request to Response ! trait Service[Request, Response] {! def apply(req: Request): Response! }! ! Equivalent to Scala’s Function1: Request=>Response
34. 34. Service But it has to be asynchronous, and treat exception as value ! trait Service[Request, Response] {! def apply(req: Request): Future[Response]! }! !
35. 35. Service It also has to be closable for proper management of resources. ! trait Service[Request, Response] {! def apply(req: Request): Future[Response]! def close(): Future[Unit]! }!
36. 36. ServiceFactory Factory of Service (Function that returns a Function)
37. 37. ServiceFactory Function that asynchronously returns a Service ! trait ServiceFactory[Req, Rep] {! def apply(): Future[Service[Req, Rep]]! def close(): Future[Unit]! }
38. 38. Agenda 1. Motivation 2.Finagle TL;DR 3.Three Key Abstractions 4.Finagle Big Picture
39. 39. Transport trait Transport[In, Out] { def write(req: In): Future[Unit] def read(): Future[Out] } Transport
40. 40. Dispatcher class Dispatcher[A, B] extends Service[…] { def apply(req: A): Future[B] } Dispatcher Transport
41. 41. Connection Pool class ConnPool[A,B] extends ServiceFactory[…]{ def apply(): Future[Service[A, B]] } Connection Dispatcher Transport Pool
42. 42. Load Balancer class LoadBalancer[A,B] extends ServiceFactory[…]{ def apply(): Future[Service[A, B]] } Connection Dispatcher Transport Pool Load Balancer
43. 43. Finagle in Action Connection Dispatcher Transport Pool Load Balancer val req: Request loadbalancer() // LB select an host // Pool select a connection
44. 44. Finagle in Action val req: Request loadbalancer() flatMap { service => // LBService(PoolService(disp)) } Connection Dispatcher Transport Pool Load Balancer
45. 45. Finagle in Action val req: Request loadbalancer() flatMap { service => // LBService(PoolService(disp)) service(req) // disp write req on Transp. } Connection Dispatcher Transport Pool Load Balancer
46. 46. Finagle in Action val req: Request loadbalancer() flatMap { service => // LBService(PoolService(disp)) service(req) ensure { service.close() // close LBService // close PoolService } } Connection Dispatcher Transport Pool Load Balancer
47. 47. FactoryToService class FactoryToService[A, B](factory: ServiceFactory[A, B]) extends Service[A, B] { def apply(request: A): Future[B] = factory() flatMap { service => service(request) ensure { service.close() } } } Connection Dispatcher Transport Pool Load Balancer FactoryTo Service
48. 48. Connection Dispatcher Transport Pool Load Balancer Finagle is “just” a composition of independent functions on top of a Transport FactoryTo Service
49. 49. The “Simplified” Full Picture Stats Timeout Draining Load Balancer Monitor Stats Failure accrual Timeout Conn. Pool Expiration Dispatcher ServiceFactory Service
50. 50. The Stack is configured for you You rarely have to configured the stack, we have a series of predefined stack for you. ! scala> val client: ServiceFactory[Req, Res] = Http.client! .newClient(“localhost:8080")! !
51. 51. The Stack is configured for you You rarely have to configured the stack, we have a series of predefined stack for you. ! scala> val client: Service[Req, Res] = Http.client! .newClient(“localhost:8080")! .toService! scala> client(req)!
52. 52. The Stack is configured for you You rarely have to configured the stack, we have a series of predefined stack for you. ! scala> val client: Service[Req, Res] = Http.client! .configured(DefaultPool.Param(5, 10, 0, 1.minute, 100))! .newClient(“localhost:8080")! .toService! scala> client(req)! ! You can configure any layer of the Stack
53. 53. Finagle is doing more… • Load balancing • Connection pooling and request buffering • Dynamic membership (Zookeeper) • Failure detection and mitigation (fail-fast & failure accrual) • Statistics for visibility • Distributed tracing (Zipkin) • Cancellation propagation • Automatic retrying • Graceful shutdown and request draining • GC avoidance - traffic shaping • Backup requests
55. 55. Load balancing 1 0 0 10
56. 56. Load balancing 1 10 0
57. 57. Load balancing 24 25 22 0
58. 58. A Better Load Balancer Key idea: Use history of recorded latencies to estimate the “cost” of sending a request to a server
59. 59. Experimental Latencies
60. 60. A better Load Balancer First idea: calculate an average of the latencies
61. 61. Experimental Latencies
62. 62. A better Load Balancer Better idea: calculate EWMA average of the latencies EWMA(n+1) = EWMA(n) * ⍺ + (1 - ⍺) * Xn+1
63. 63. Experimental Latencies
64. 64. A better Load Balancer The devil is in the details ! Recency of data: EWMA unevenly spaced t-Series EWMA(n+1) = EWMA(n) * e-td/Tau + (1 - e-td/Tau) * Xn+1 No history: Initial cost=0 + Probation
65. 65. Cost & Experimental Latencies
66. 66. A Better Load Balancer The devil is in the details ! Speed of convergence: Cost “rides the peaks” Outliers: Decaying cost
67. 67. Cost & Experimental Latencies
68. 68. A Better Load Balancer The devil is in the details ! CPU intensive: “The Power of 2 Choices”
69. 69. A Better Load Balancer “The Power of 2 Choices in Randomized Load Balancing” P.h.D Thesis of M. Mitzenmacher
70. 70. Load balancing 0 20 0 10 0 cost: 20ms cost: 10ms cost: 10ms
71. 71. cost: 18ms Load balancing 0 20 0 10 0 20ms cost: 10ms cost: 10ms
72. 72. The “Simplified” Full Picture Stats Timeout Draining Load Balancer Monitor Stats Failure accrual Timeout Conn. Pool Expiration Dispatcher ServiceFactory Service
73. 73. Finagle’s Architecture Experience from writing “large scale” software ! Independent modules Simple contracts Powerful combinators
74. 74. Conclusion • Finagle is a library that lets you treat RPC as functions • Finagle is itself composed of independent functions • Very widely used at Twitter, Foursquare, Tumblr, Pinterest, SoundCloud… • Protocol agnostic (thrift, http, redis, mysql…) Github: github.com/twitter/finagle “Your server as a Function” http://monkey.org/~marius/ funsrv.pdf
75. 75. @finagle @stevegury