Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Scaling Distributes Systems
Natali...
InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese an...
Presented at QCon New York
www.qconnewyork.com
Purpose of QCon
- to empower software development by facilitating the sprea...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Who am I?
2011: Received PhD degre...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Sources
Research findings
Experienc...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Scaling a Sysem
Scaling ALL aspect...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Scaling a Sysem
Scaling ALL aspect...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Scaling on language level
Actor mo...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Language – Actor Model
Built-in co...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Language – Functional programming
...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Fault Tolerance
105 cores – approx...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Philosophy
Principles
Ideas
Core v...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
RELEASE Aim
To scale the radical a...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
RELEASE Aim
To scale the radical a...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
RELEASE Aim
To scale the radical a...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
RELEASE Aim
To scale the radical a...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Typical Target Architecture - 105
...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Erlang Overview
Erlang
is a functi...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Distributed Erlang
Distributed Erl...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Distributed Erlang
Transitive conn...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Distributed Erlang
Transitive conn...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Distributed Erlang Scalability Lim...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Distributed Erlang Scalability Lim...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Distributed Erlang Scalability Lim...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Distributed Erlang Scalability Lim...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Distributed Erlang Scalability Lim...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
SD Erlang
Network Scalability
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
S group Operational Semantics
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
S group Operational Semantics
Vali...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Ongoing and Future Work
S groups
I...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Sources
SD Erlang http://www.dcs.g...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
Thank you!
N. Chechina, RELEASE te...
Distributed Erlang
Scalable Distributed (SD) Erlang
Operational Semantics
Plans
Sources
J. Armstrong.
Erlang.
Commun. ACM,...
Watch the video with slide synchronization on
InfoQ.com!
http://www.infoq.com/presentations/actor-
functional-scale
Prochain SlideShare
Chargement dans…5
×

Scaling Distributed Systems

1 322 vues

Publié le

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1VAmi5s.

Natalia Chechina outlines features of actor and functional programming models, and the reason these models attract so much interest in parallel, concurrent, and scaling world. She talks about fault tolerance, its importance in large scale systems, and the approaches to implement it. Filmed at qconnewyork.com.

Natalia Chechina is a Research Fellow at the University of Glasgow. She received her PhD degree from Heriot-Watt University, UK in 2011. She then was a Research Associate at the University of Glasgow leading Scalable Distributed Erlang work package (WP3) in the RELEASE project.

Publié dans : Technologie
  • Soyez le premier à commenter

Scaling Distributed Systems

  1. 1. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Scaling Distributes Systems Natalia Chechina and RELEASE Team June 11, 2015 N. Chechina, RELEASE team Scaling Distributes Systems
  2. 2. InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /actor-functional-scale
  3. 3. Presented at QCon New York www.qconnewyork.com Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide
  4. 4. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Who am I? 2011: Received PhD degree in Computer Science from Heriot-Watt University, UK 2011-2015: WP3 lead in the EU RELEASE Project at Glasgow University, UK March 2015: Research Fellow at Glasgow University, UK Main research interest: Scaling distributed computations on commodity hardware N. Chechina, RELEASE team Scaling Distributes Systems
  5. 5. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Sources Research findings Experience from the RELEASE project Funded by EU FP7 Framework 5 academic & 3 industrial partners Aim: To scale the radical actor (concurrency-oriented) paradigm to build reliable general-purpose software, such as server-based systems, on massively parallel machines (105 cores) Erlang programming language Experience of other researches and developers N. Chechina, RELEASE team Scaling Distributes Systems
  6. 6. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Scaling a Sysem Scaling ALL aspects of computation Application Language Virtual Machine In-memory data structures Persistent data structures Tools (debugging, monitoring, etc) N. Chechina, RELEASE team Scaling Distributes Systems
  7. 7. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Scaling a Sysem Scaling ALL aspects of computation Application Language Virtual Machine In-memory data structures Persistent data structures Tools (debugging, monitoring, etc) N. Chechina, RELEASE team Scaling Distributes Systems
  8. 8. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Scaling on language level Actor model Functional programming N. Chechina, RELEASE team Scaling Distributes Systems
  9. 9. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Language – Actor Model Built-in concurrency Actors have own states and don’t share them Communication between actor happens only via message passing Actors can spawn new actors N. Chechina, RELEASE team Scaling Distributes Systems
  10. 10. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Language – Functional programming Fundamental operation – application of functions to arguments Higher-order functions – well-structured software Modules – independent, reusable Lazy evaluations Variables given values only once N. Chechina, RELEASE team Scaling Distributes Systems
  11. 11. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Fault Tolerance 105 cores – approx. failure of 1 core per hour Non-defensive approach – Supervision & ”Let it crash” N. Chechina, RELEASE team Scaling Distributes Systems
  12. 12. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Philosophy Principles Ideas Core values N. Chechina, RELEASE team Scaling Distributes Systems
  13. 13. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources RELEASE Aim To scale the radical actor (concurrency-oriented) paradigm to build reliable general-purpose software, such as server-based systems, on massively parallel machines (105 cores). N. Chechina, RELEASE team Scaling Distributes Systems
  14. 14. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources RELEASE Aim To scale the radical actor (concurrency-oriented) paradigm to build reliable general-purpose software, such as server-based systems, on massively parallel machines (105 cores). Erlang N. Chechina, RELEASE team Scaling Distributes Systems
  15. 15. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources RELEASE Aim To scale the radical actor (concurrency-oriented) paradigm to build reliable general-purpose software, such as server-based systems, on massively parallel machines (105 cores). Erlang VM aspects, e.g. synchronisation on internal data structures Language aspects, e.g. maintaining a fully connected network of nodes, explicit process placement Tool support N. Chechina, RELEASE team Scaling Distributes Systems
  16. 16. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources RELEASE Aim To scale the radical actor (concurrency-oriented) paradigm to build reliable general-purpose software, such as server-based systems, on massively parallel machines (105 cores). Erlang VM aspects, e.g. synchronisation on internal data structures Language aspects, e.g. maintaining a fully connected network of nodes, explicit process placement Tool support N. Chechina, RELEASE team Scaling Distributes Systems
  17. 17. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Typical Target Architecture - 105 cores Commodity hardware Non-uniform communication (Level0 – same host, Level1 – same cluster, etc) N. Chechina, RELEASE team Scaling Distributes Systems
  18. 18. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Erlang Overview Erlang is a functional general purpose concurrent programming language developed in 1986 at Ericsson is dynamically typed was designed for distributed, fault-tolerant, massively concurrent, and soft-real time systems follows let it crash and share nothing philosophy The language primitives are processes. Erlang concurrency is handled by the language and not by the operating system [Arm10]. N. Chechina, RELEASE team Scaling Distributes Systems
  19. 19. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Distributed Erlang Distributed Erlang N. Chechina, RELEASE team Scaling Distributes Systems
  20. 20. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Distributed Erlang Transitive connections N. Chechina, RELEASE team Scaling Distributes Systems
  21. 21. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Distributed Erlang Transitive connections Explicit Placement, i.e. spawn(Node, Module, Function, Args) → pid() N. Chechina, RELEASE team Scaling Distributes Systems
  22. 22. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Distributed Erlang Scalability Limitations Global operations Global operations, i.e. registering names using global module N. Chechina, RELEASE team Scaling Distributes Systems
  23. 23. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Distributed Erlang Scalability Limitations Global operations Global operations, i.e. registering names using global module Other global operations, e.g. using rpc:call to call multiple nodes N. Chechina, RELEASE team Scaling Distributes Systems
  24. 24. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Distributed Erlang Scalability Limitations Global operations Global operations, i.e. registering names using global module Other global operations, e.g. using rpc:call to call multiple nodes All-to-all transitive connections N. Chechina, RELEASE team Scaling Distributes Systems
  25. 25. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Distributed Erlang Scalability Limitations Global operations Global operations, i.e. registering names using global module Other global operations, e.g. using rpc:call to call multiple nodes All-to-all transitive connections But... aren’t global operations and transitivity are optional in distributed Erlang? Why use them if they are a bottleneck? N. Chechina, RELEASE team Scaling Distributes Systems
  26. 26. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Distributed Erlang Scalability Limitations Global operations Global operations, i.e. registering names using global module Other global operations, e.g. using rpc:call to call multiple nodes All-to-all transitive connections But... aren’t global operations and transitivity are optional in distributed Erlang? Why use them if they are a bottleneck? Reliability and fault tolerance – when a process or a node fail, the remaining nodes know about that. The same holds for the recovery It’s already there – no extra effort to connect nodes and distribute information Easy to scale – a new node knows about running nodes, and vice versa N. Chechina, RELEASE team Scaling Distributes Systems
  27. 27. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Scalable Distributed (SD) Erlang SD Erlang is a small conservative extension of Distributed Erlang Network Scalability All-to-all connections are not scalable onto 1000s of nodes Aim: Reduce connectivity Semi-explicit Placement Becomes not feasible for a programmer to be aware of all nodes Aim: Automatic process placement in groups of nodes N. Chechina, RELEASE team Scaling Distributes Systems
  28. 28. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Free Node Connections vs. S group Node Connections N. Chechina, RELEASE team Scaling Distributes Systems
  29. 29. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Free Node Connections vs. S group Node Connections N. Chechina, RELEASE team Scaling Distributes Systems
  30. 30. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Free Node Connections vs. S group Node Connections N. Chechina, RELEASE team Scaling Distributes Systems
  31. 31. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Free Node Connections vs. S group Node Connections N. Chechina, RELEASE team Scaling Distributes Systems
  32. 32. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Free Node Connections vs. S group Node Connections N. Chechina, RELEASE team Scaling Distributes Systems
  33. 33. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Connections between Different Types of Nodes N. Chechina, RELEASE team Scaling Distributes Systems
  34. 34. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Connections between Different Types of Nodes N. Chechina, RELEASE team Scaling Distributes Systems
  35. 35. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Connections between Different Types of Nodes N. Chechina, RELEASE team Scaling Distributes Systems
  36. 36. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Connections between Different Types of Nodes N. Chechina, RELEASE team Scaling Distributes Systems
  37. 37. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Connections between Different Types of Nodes N. Chechina, RELEASE team Scaling Distributes Systems
  38. 38. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Connections between Different Types of Nodes N. Chechina, RELEASE team Scaling Distributes Systems
  39. 39. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Why S groups? Preserve Erlang phylosophy & transitivity and scale Considered approaches Grouping nodes according to their hash values A hierarchical approach Overlapping s groups Other approaches Distributed Erlang global groups Spapi Router (SpilGames) Custom routing on non-transtively connected normal or hidden nodes N. Chechina, RELEASE team Scaling Distributes Systems
  40. 40. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Hierarchical Grouping N. Chechina, RELEASE team Scaling Distributes Systems
  41. 41. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Free Nodes and S groups N. Chechina, RELEASE team Scaling Distributes Systems
  42. 42. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Embedded Grouping N. Chechina, RELEASE team Scaling Distributes Systems
  43. 43. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement SD Erlang Improves Scalability N. Chechina, RELEASE team Scaling Distributes Systems
  44. 44. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Speed Up of Distributed Erlang Orbit & SD Erlang Orbit N. Chechina, RELEASE team Scaling Distributes Systems
  45. 45. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Speed Up of Distributed Erlang ACO & SD Erlang ACO N. Chechina, RELEASE team Scaling Distributes Systems
  46. 46. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Semi-Explicit Placement Communication latencies between nodes may vary according to their relative positions In terms of communication time nodes may be “nearby” or “far away” We may wish some tasks to be close together because they’re communicating with each other a lot N. Chechina, RELEASE team Scaling Distributes Systems
  47. 47. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Example System structure N. Chechina, RELEASE team Scaling Distributes Systems
  48. 48. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Example: system structure Racks N. Chechina, RELEASE team Scaling Distributes Systems
  49. 49. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Example: system structure Clusters N. Chechina, RELEASE team Scaling Distributes Systems
  50. 50. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Example: system structure Cloud N. Chechina, RELEASE team Scaling Distributes Systems
  51. 51. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Dendrogram bwlf16 bwlf13 bwlf15 bwlf03 bwlf05 bwlf34 bwlf06 bwlf12 bwlf08 bwlf27 bwlf33 bwlf18 bwlf28 bwlf23 bwlf04 bwlf07 bwlf10 bwlf17 bwlf26 bwlf01 bwlf20 bwlf21 bwlf31 bwlf14 bwlf19 bwlf24 bwlf25 bwlf29 bwlf02 bwlf09 bwlf22 bwlf30 bwlf11 bwlf32 amaterasu persephone oberon cantor osiris 1000015000200002500030000 Height N. Chechina, RELEASE team Scaling Distributes Systems
  52. 52. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Measuring communication distance We can define a distance function d on the set V of Erlang VMs in a distributed system by d(x, y) = 0 if x = y 2− (x,y) if x = y. where (x, y) is the length of the longest path which is shared by the paths from the root to x and y. N. Chechina, RELEASE team Scaling Distributes Systems
  53. 53. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Distances (b, c) = 2 d(b, c) = 2−2 = 1/4 N. Chechina, RELEASE team Scaling Distributes Systems
  54. 54. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Distances (b, g) = 1 d(b, g) = 2−1 = 1/2 N. Chechina, RELEASE team Scaling Distributes Systems
  55. 55. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement Distances (b, k) = 0 d(b, k) = 2−0 = 1 N. Chechina, RELEASE team Scaling Distributes Systems
  56. 56. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources SD Erlang Network Scalability Validation Semi-Explicit Placement choose nodes/1 Every node may have a list of attributes choose nodes/1 function returns a list of nodes that satisfy given restrictions s_group: choose_nodes ([Parameter]) -> [Node] where Parameter = {s_group , SGroupName} | {attribute , AttributeName } | {nearer , 0.4} | {between , 0.5, 0.7} SGroupName = group_name () AttributeName = term () N. Chechina, RELEASE team Scaling Distributes Systems
  57. 57. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources S group Operational Semantics Validation of SD Erlang Semantics and Implementation Operational Semantics (state, command, ni) −→ (state , value) Executing command on node ni in state returns value and transitions to state . N. Chechina, RELEASE team Scaling Distributes Systems
  58. 58. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources S group Operational Semantics Validation of SD Erlang Semantics and Implementation Validation of Semantics and Implementation Validate the consistency between the formal semantics and the SD Erlang implementation Use Erlang QuickCheck tool developed by QuviQ Behaviour is specified by properties expressed in a logical form eqc statem is a finite state machine in QuickCheck Figure: Testing SD Erlang Using QuickCheck eqc statem N. Chechina, RELEASE team Scaling Distributes Systems
  59. 59. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Ongoing and Future Work S groups Introduce more patterns, for example, routing for a tree structure Analysis of fault tolerance strategies and features in SD Erlang applications Semi-explicit Placement Discovering system structure at runtime Robustness – dynamically adjusting a view of the system if new nodes join it, or if existing ones fail N. Chechina, RELEASE team Scaling Distributes Systems
  60. 60. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Sources SD Erlang http://www.dcs.gla.ac.uk/research/sd-erlang/ RELEASE Project http://www.release-project.eu/ Deployment tool Wombat https://www.erlang-solutions.com/products/wombat Profiling tools Percept2 https://github.com/release-project/percept2 devo https://www.youtube.com/watch?v=Ox30TBDcFPw Benchmarking BenchErl http://release.softlab.ntua.gr/bencherl/index.html DEbench, Orbit, ACO https://github.com/release-project/benchmarks N. Chechina, RELEASE team Scaling Distributes Systems
  61. 61. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources Thank you! N. Chechina, RELEASE team Scaling Distributes Systems
  62. 62. Distributed Erlang Scalable Distributed (SD) Erlang Operational Semantics Plans Sources J. Armstrong. Erlang. Commun. ACM, 53:68–75, 2010. N. Chechina, RELEASE team Scaling Distributes Systems
  63. 63. Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations/actor- functional-scale

×