Engineering resilient distributed systems remains extremely challenging, particularly in mapping from collective specifications to individual device behavior. Aggregate programming aims to address this problem using a computational field abstraction to provide inherent guarantees of resilience, scalability, and safe composition. These capabilities are provided, however, by an expressive but terse set of operators too low-level for pragmatic use in complex systems development. We thus present an API to raise the level of abstraction, thereby providing an accessible and user-friendly interface for construction of complex resilient distributed systems. In particular, we capture and organize a large, heterogeneous collection of algorithms and use patterns into a unified framework, including methods for common tasks such as leader election, distance and state estimation, and gossip-based information dissemination. We demonstrate how the expressiveness of this library reduces the abstraction gap required to engineer scenarios of large-scale pervasive computing, while introducing the novel multiInstance pattern enabling an unanticipated composition of computational fields.
Towards a Foundational API for Resilient Distributed Systems Design
1. Towards a Foundational API for Resilient Distributed
Systems Design
Matteo Francia1, Danilo Pianini1, Jacob Beal2, Mirko Viroli1
1Alma Mater Studiorum—Universit`a di Bologna
2Raytheon BBN Technologies
2nd eCAS Workshop on Engineering Collective Adaptive Systems
2017-09-22 – Tucson, AZ, USA
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 1 / 27
2. Outline
1 Introduction
Device-centric engineering in the IoT era
Distributed system as a single entity
Resilient building blocks
2 A foundational library for aggregate programming
Goal and methodology
Meta patterns
Organization
Testing environment
3 Evaluation
Meet the VIP
Resource allocation
4 Conclusion
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 2 / 27
3. Introduction
Outline
1 Introduction
Device-centric engineering in the IoT era
Distributed system as a single entity
Resilient building blocks
2 A foundational library for aggregate programming
Goal and methodology
Meta patterns
Organization
Testing environment
3 Evaluation
Meet the VIP
Resource allocation
4 Conclusion
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 2 / 27
4. Introduction Device-centric engineering in the IoT era
Computing everywhere
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 3 / 27
5. Introduction Device-centric engineering in the IoT era
Device-centric engineering
Low-level details impact system design
Communication protocols
Fault tolerance
Power saving policy
...
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 4 / 27
6. Introduction Distributed system as a single entity
The aggregate programming stack
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 5 / 27
7. Introduction Distributed system as a single entity
Level 1: Field Calculus
“Everything is a field”
Stateless rounds
3 constructs
(a) State (rep x e0 e)
(b) Neighborhood (nbr x)
(c) Restriction (if c e0 e1)
H A
A
D
H
T
H
K
B
(a) rep (b) nbr
(c) if
Figure: Field calculus constructsFrancia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 6 / 27
8. Introduction Distributed system as a single entity
Protelis
A concrete implementation of field calculus
Functional paradigm
Java-family syntax
Java interoperability
P ::= I F s; ;; Program
I ::= import m ;; Imports
F ::= def f(x) {s;} ;; Fun definition
s ::= e | let x = e ;; Statement
e ::= ;; Expression
| f(e) | f.apply(e) ;; Fun call
| rep(x<-w){s;} ;; State
| if(e){s;} else {s ;};; Restriction
| nbr{s;} ;; Neighborhood
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 7 / 27
9. Introduction Resilient building blocks
Back to the stack
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 8 / 27
10. Introduction Resilient building blocks
Level 2: Building blocks
Coordination in 4 patterns:
G Spreading
C Aggregation
T Evaporation
S Clustering
With nice features:
Self-stabilization [VBDP15]
Composability [BV14]
Substitutability [VAB+17]
Similar in functionality to basic
patterns in bio-inspired
coordination [FSM+13]
(a) G: spreading (b) C: aggregation
3
1
7
2
4
3
3
1
0
(c) T: evaporation (d) S: clustering
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 9 / 27
11. A foundational library for aggregate programming
Outline
1 Introduction
Device-centric engineering in the IoT era
Distributed system as a single entity
Resilient building blocks
2 A foundational library for aggregate programming
Goal and methodology
Meta patterns
Organization
Testing environment
3 Evaluation
Meet the VIP
Resource allocation
4 Conclusion
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 9 / 27
12. A foundational library for aggregate programming Goal and methodology
Aggregate programming
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 10 / 27
13. A foundational library for aggregate programming Goal and methodology
Level 3: protelis-lang
GOALS
Collect existing algorithms
Classify them by scope
Fill in missing patterns
Encapsulate coordination best practices “under the hood”
Two-stages:
1 Search in existing literature
2 Compare with existing abstractions and technologies
[FSM+13, BDU+13]
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 11 / 27
14. A foundational library for aggregate programming Meta patterns
multiInstance
A
A
B
B
Idea
Running multiple copies of a process in parallel, one for each identified
“source” and aggregate their outputs
This pattern can be leveraged for several applications:
1 Temporal replication [PBV16];
2 acquire information coming from a dynamic number of sources;
3 extend the application scope of existing algorithms.
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 12 / 27
15. A foundational library for aggregate programming Organization
Library structure
Specialized building blocks
Plus meta-programming
protelis
coord state
time
countDown
cyclicTimer
limitedMemory
T
...
meta
boundSpreading
multiInstance
multiRegion
publishSubscribe
...
sparsechoice
S
spreading
broadcast
crfGradient
distanceTo
distanceBetween
flexGradient
gradcast
G
...
accumulation
average
C
cMultiSum
cMultiMin
consensus
countDevices
...
nonselfstabilizing
accumulation
gossip
...
nonselfstabilizing
time
gossipEver
...
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 13 / 27
16. A foundational library for aggregate programming Organization
Code snippet
module protelis:coord:spreading
/** Propagation of information from a source.
* @param source bool , whether the device is a source
* @param initial T, information spread by the source
* @param metric () -> num , how to estimate distance
* @param accumulate (T) -> T, how to accumulate information
* @return T, accumulated value
*/
def G(source , initial , metric , accumulate) {
/* implementation with rep , nbr , if ... */
}
/** Distance to the closest source.
* @param source bool , whether the device is a source
* @return num , field distances to the closest source
*/
def distanceTo(source) {
G(source , 0, nbrRange , (v) -> { v + nbrRange.apply () })
}
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 14 / 27
17. A foundational library for aggregate programming Testing environment
Automated testing I
Testing is a serious concern, as we need to:
verify the correctness of the implementation
prevent regressions
Most of the algorithms in protelis-lang are distributed
testing locally provides information of little use
a simulated environment is required
We exploited Alchemist [PMV13] for testing on simulated networks
Every test is composed of:
An Alchemist YAML specification, commented with the expected result
A Protelis source file
All the provided functions are tested
If a bug is found and solved, a regression test is included
Testing happens at every build
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 15 / 27
19. A foundational library for aggregate programming Testing environment
Testing example: distanceTo
Stabilization: network converges to a stable state
Self-Stabilization: network converges after perturbations
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 17 / 27
20. Evaluation
Outline
1 Introduction
Device-centric engineering in the IoT era
Distributed system as a single entity
Resilient building blocks
2 A foundational library for aggregate programming
Goal and methodology
Meta patterns
Organization
Testing environment
3 Evaluation
Meet the VIP
Resource allocation
4 Conclusion
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 17 / 27
21. Evaluation
Evaluation in crowded scenarios
Two scenarios with 1000 devices each
Meet the VIP: steering people towards POIs with fixed capacity
Resource allocation: coordination of emergency responders to
people requesting aid
Comparing functionality and complexity of two implementations
BB “Building-blocks-only” solution not using any library code
Not allowed to use any code from the library
Self stabilizing based on the G/C/T/S blocks
L Solution with library code (L), including multiInstance
Must improve over BB by using multiInstance
Allowed to use any of function of protelis-lang to reduce code
complexity
To reproduce them: https://bitbucket.org/mfrancia/2017-ecas-experiments
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 18 / 27
22. Evaluation Meet the VIP
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 19 / 27
23. Evaluation Meet the VIP
Meet the VIP I
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 20 / 27
24. Evaluation Meet the VIP
Meet the VIP II
Improved functionality
0 5 10 15 20 25
Simulation time (mins)
0
200
400
Devices
Asymmetric - attendees to POI Thr
L-A
L-B
L-C
BB-A
BB-B
BB-C
BB can only allocate attendees to the closest POI, resulting in
over-crowded areas
underutilized areas
L enables to count attendees in each POI, allowing a balanced and fair
allocation
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 21 / 27
25. Evaluation Meet the VIP
Meet the VIP III
Complexity comparison
BB is 8 lines of code
L is 19 lines of code
With 11 more lines of code, L overcomes BB’s partitioning issues and
enables novel capabilities
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 22 / 27
27. Evaluation Resource allocation
Resource Allocation II
Improved functionality
0 5 10 15 20
Simulation time (mins)
0.0
2.5
5.0
Resources
Asymmetric - resources serving needs L-A
L-B
L-C
L-D
BB-A
BB-B
BB-C
BB-D
BB allocates responders only to the closest need
L is capable of a more complex allocation, for instance:
Responders are allocated to the emergency with the least allocated
resources until its needs are satisfied
If two emergencies have the same amount of resources, responders are
allocated to the closest one
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 24 / 27
28. Evaluation Resource allocation
Resource Allocation III
Complexity comparison
BB is 29 lines of code
L is 17 lines of code
Here L achieves a better allocation policy in fewer lines of code
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 25 / 27
29. Conclusion
Outline
1 Introduction
Device-centric engineering in the IoT era
Distributed system as a single entity
Resilient building blocks
2 A foundational library for aggregate programming
Goal and methodology
Meta patterns
Organization
Testing environment
3 Evaluation
Meet the VIP
Resource allocation
4 Conclusion
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 25 / 27
30. Conclusion
Conclusion
With more than 150 functions, protelis-lang:
offers a systematic coverage of the state-of-the art of coordination
abstractions;
Many distributed functions are usable out of the box
Reduces the abstraction gap when engineering aggregate programs;
Less lines of code for the same application
Introduces multiInstance.
Overcomes the limitations of the building blocks
Paves the way to a novel family of meta patterns
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 26 / 27
31. Conclusion
References I
Jacob Beal, Stefan Dulman, Kyle Usbeck, Mirko Viroli, and Nikolaus Correll.
Organizing the aggregate: Languages for spatial computing.
In Marjan Mernik, editor, Formal and Practical Aspects of Domain-Specific Languages:
Recent Developments, chapter 16, pages 436–501. IGI Global, 2013.
A longer version available at: tt http://arxiv.org/abs/1202.5509.
Jacob Beal and Mirko Viroli.
Building blocks for aggregate programming of self-organising applications.
In Eighth IEEE International Conference on Self-Adaptive and Self-Organizing Systems
Workshops, SASOW 2014, London, United Kingdom, September 8-12, 2014, pages 8–13,
2014.
Jose Luis Fernandez-Marquez, Giovanna Di Marzo Serugendo, Sara Montagna, Mirko
Viroli, and Josep Llu´ıs Arcos.
Description and composition of bio-inspired design patterns: a complete overview.
Natural Computing, 12(1):43–67, 2013.
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 26 / 27
32. Conclusion
References II
Danilo Pianini, Jacob Beal, and Mirko Viroli.
Improving gossip dynamics through overlapping replicates.
In Coordination Models and Languages - 18th IFIP WG 6.1 International Conference,
COORDINATION 2016, Held as Part of the 11th International Federated Conference on
Distributed Computing Techniques, DisCoTec 2016, Heraklion, Crete, Greece, June 6-9,
2016, Proceedings, pages 192–207, 2016.
Danilo Pianini, Sara Montagna, and Mirko Viroli.
Chemical-oriented simulation of computational systems with ALCHEMIST.
J. Simulation, 7(3):202–215, 2013.
Mirko Viroli, Giorgio Audrito, Jacob Beal, Ferruccio Damiani, and Danilo Pianini.
Engineering resilient collective adaptive systems by self-stabilisation.
ACM Transactions on Modeling and Computer Simulation (TOMACS), 2017.
Submitted for publication.
Mirko Viroli, Jacob Beal, Ferruccio Damiani, and Danilo Pianini.
Efficient engineering of complex self-organising systems by self-stabilising fields.
In 2015 IEEE 9th International Conference on Self-Adaptive and Self-Organizing Systems,
Cambridge, MA, USA, September 21-25, 2015, pages 81–90, 2015.
Francia, Pianini, Beal, Viroli (UniBo, BBN) protelis-lang 2017-09-18 – eCAS 27 / 27