1. Tutorial:
Programming Elasticity in the Cloud
Hong-Linh Truong
Distributed Systems Group
Vienna University of Technology
truong@dsg.tuwien.ac.at
http://dsg.tuwien.ac.at/research/viecom/
Cloudcom 2014, 15 Dec, 2014,
Singapore
1
The 6th International Conference on Cloud Computing Technology and Science

2. Distributed Systems Group@TU Wien
 ~ 30 people, 1 full prof., 1 Priv-Doz, 4 post-docs,
16 phd students, ~14 nationalities
 Working on cloud computing, elastic computing,
SOC/SOA, IoT, internet engineering, service
analytics, social computing, and process
management.
Cloudcom 2014, 15 Dec, 2014,
Singapore
2

3. Joint work with:
Schahram Dustdar, Muhammad Candra, Georgiana Copil, Duc-
Hung Le, Daniel Moldovan, Stefan Nastic, Tien-Dung Nguyen,
Anindita Sarkar, Ognjen Scekic, Philipp Zeppezauer
Cloudcom 2014, 15 Dec, 2014,
Singapore
3
NOTE: The content includes some ongoing work
Acknowledgements

4. Outline
 Part 1 – Elastic Computing Principles and
Models
 Motivation scenario, Elastic principles and the
Vienna Elasticity Computing Model
 Part 2 – Techniques for programming and
eingeering elasticity for native cloud services
 Configuration, deployment, monitoring, and control
with COMOT and its Salsa, SYBL, MELA
 Part 3
 Hand-on exercise
 Summary
Cloudcom 2014, 15 Dec, 2014,
Singapore
4

5. PART 1: ELASTIC COMPUTING
- PRINCIPLES AND MODELS
Cloudcom 2014, 15 Dec,
2014, Singapore
5

6. Some 50 billion devices and sensors exist for M2M applications
IoT and Cloud Computing enable smart services ecosystem and
collaboration opportunities
Managed services
• Portfolio
management
• Event management
• Analytics
Provisioning
• Services
• SIM profile
configuration
• Network
configuration
Controls
• Activation
• Deactivation
• Privacy
• Security
Transaction Mgmt.
• Visibility
• Billing
• Reporting
Integration
framework
Algorithm
Chart engine
builder
Predictiv
e
modeling
Incidents
manager
Expert rule
engine
FDD Service
Mgmt
Storage
policies
Databas
e
Operations manger
manager
Portfolio
Mgmt Analyic
s
engine
Blackbo
x
module
Location
awarenes
s
GUI
builde
r
Event
mgmt
Data
mining
Resource
mgmt.
Regressio
n engine
Open
integratio
n platform
Resource
manager
Point
metering
framewor
k Numerous Forms Of Smart Services…
Access
control
Environment
Compliance
Street Light
Management
Food Transfer
Process
Public
Safety
Industrial
process
parameter
s
Parking
Control
Waste
Management
Facilities
Control
HealthCare
Power
Quality Control
Lighting
Control
KIOSK
Monitoring
CCTV
Monitoring
Hospitality Sector Healthcare Sector
Transport Sector Education Sector
Datacenters
Government Sector
Industrial Sector
Finance Sector
Utilities and Smart Grid
Airports, ports and
Critical Infrastructure
Ubiquitous Managed Services Solution Across Business Verticals
Cloudcom 2014, 15 Dec,
2014, Singapore
6

7. Motivation
 City-scale proactive and predictive maintenance
 What do we need
 Sensors and actuators
 IoT cloud platforms: gateways/VMs, complex event
processing, message-oriented middleware, etc.
 (Big/smart) data analytics services and storages
 Human experts
 Many stakeholders
 Cloud provider, maintenance operators, IoT platform
developers/providers, data analytics developer/provider,
domain experts, etc.
Cloudcom 2014, 15 Dec, 2014,
Singapore
7

8. Scenario
Cloudcom 2014, 15 Dec, 2014,
Singapore
8
Offers services for
handling IoT Data
Offers services for big,
data analytics
Offers services for
complex problem solving
using human experts
IoT Cloud Platform
Data Analytics
Platform
Expert Provisioning
Platform
Sensors
<<send data>>
<<analyze data>> <<notify possible
problem>>
<<control/configure
sensors>>
Predictive maintenance company
<<monitor>>
Chillers
<<predict and solve
problems>>
<<control
services>>
<<control
algorithms>>

9. Observations
 Data resources
 Activate/change sensor deployment/configurations for
required data; changing types of data sources for analytics
 Cloud platform services
 Deploy/reconfigure cloud services handling changing data
 Data analytics
 Switch and combine different types of data analytics
processes and engines due to the severity of problems and
quality of results
 Teams of human experts
 Forming and changing different configurations of teams
during the specific problems and problem severity.
Cloudcom 2014, 15 Dec, 2014, 9
Singapore

10. Elasticity in computing – broad view
1. Demand elasticity
Elastic demands from consumers
2. Output elasticity
Multiple outputs with different price and quality
3. Input elasticity
Elastic data inputs, e.g., deal with opportunistic data
4. Elastic pricing and quality models associated
resources
Cloudcom 2014, 15 Dec, 2014,
Singapore
10

11. Diverse types of elasticity requirements
 Application user: “If the cost is greater than 800 Euro, there should be
a scale-in action for keeping costs in acceptable limits”
 Software provider: “Response time should be less than amount X
varying with the number of users.”
 Developer: “The result from the data analytics algorithm must reach a
certain data accuracy under a cost constraint. I don’t care about how
many resources should be used for executing this code.”
 Cloud provider: “When availability is higher than 99% for a period of
time, and the cost is the same as for availability 80%, the cost should
increase with 10%.”
Cloudcom 2014, 15 Dec, 2014,
Singapore
11

12. Complex dependencies in elasticity
 More data is needed only
when necessary
 More data  more
computational resources
(e.g. more VMs)
 More types of data 
different types of analytics
processes  different
computational models
 Change quality of results
 quality of data
 response time
 cost
 types of result (form of
the data output, e.g.
tree, visual, story, etc.)
12
Data
Computational
Model
Analytics
Process
Analytics Result
Data
Data
Datax
Datay
Dataz
Computational
Modelx
Computational
Model y
Computationyal
Modelz
Computational
Analytics
Process
AnalytPicrsocess
Process
Analytics
AnalyPtricoscess
Process
Analytics
Quality of
Result
Cloudcom 2014, 15 Dec, 2014,
Singapore

13. Elasticity with multiple
types of resources
Machine-based
Computing
Human-based
Computing
Things-based
computing
Grid
Processing
Unit
Comm. Architecture
SMP
S. Dustdar, H. Truong, “Virtualizing Software and Humans
for Elastic Processes in Multiple Clouds – a Service
Management Perspective”, in International Journal of Next
Generation Computing, 2012
Ad hoc networks Web of things
Cloudcom 2014, 15 Dec, 2014,
Singapore
13

14. SO HOWDO WESUPPORTTHESEISSUESIN A SYSTEMATICALWAY?
Cloudcom 2014, 15 Dec, 2014,
Singapore
14

15. Resource elasticity
Software / human-based
computing elements,
multiple clouds
Quality elasticity
Non-functional parameters e.g.,
performance, quality of data,
service availability, human
trust
Costs & Benefit
elasticity
rewards, incentives
Elasticity
Multi-dimensional Elasticity
Cloudcom 2014, 15 Dec, 2014,
Singapore
15

16. The Vienna Elastic Computing
Model
 Multi-dimensional
Elasticity
 Service computing
models
 Cloud provisioning
models
Schahram Dustdar, Hong Linh Truong: Virtualizing Software and Humans for Elastic Processes in Multiple Clouds- a Service
Management Perspective. IJNGC 3(2) (2012)
Cloudcom 2014, 15 Dec, 2014,
Singapore
16

17. Vienna Elastic Computing Model
 Multi-dimensional elasticity
 Resource, quality, cost and benefits
 Elasticity in hybrid systems of human-based,
things-based and software-based computing
resources
 Software, things and human capabilities as computing
resources in multi clouds
 End-to-end approach
 For science and business complex applications
dsg.tuwien.ac.at/research/viecom
Cloudcom 2014, 15 Dec, 2014,
Singapore
17

18. Unified elastic service unit model
for Things, Software and People
Cloudcom 2014, 15 Dec, 2014,
Singapore
18
Modeling type of
units (e.g.,
computation, data,
monitor,) and their
dependencies
Consumption,
ownership, provisioning,
price, etc.
Elastic
Service
Unit
Service
model
Unit
Dependency
Elastic
Capability
Function
The functional
capability of the unit
and interface to
access the function
Capabilities to be elastic
under different
requirements
Software People
VolunteersProfessionals
Thing
Resources

19. Things: software-Defined IoT Units
 Virtualizing IoTs resources under “service
units” with software-defined API for
accessing, configuring and controlling units
 Composing and creating gateways and
virtual topologies (of multiple gateways)
 Provisioning (atomic and composite) units
dynamically and on-demand in cloud and
edge computing environments
 Controlling elasticity based on resource,
quality and cost via software-defined APIs
Software-defined
IoT Unit
Functional API
Utility
cost-function
IoT resource and functionality binding
Late-bound
policies
Infrastructure capabilities
Governance API
Dependency
units
Provisioning API
Runtime
mechanisms
Runtime
controllers
(e.g, elasticity)
Non-functional aspects
Runtime composition
Functional aspects
Atomic software-defined IoT units
Custom
proc. logic
IoT data
storage
Communication
In-memory
image
VPN
Messaging
Sand
box
Network
overlay Protocol Volatile History
Key/Value
store
Security
Data
quality
Outliers
filter
IoT compute
GW
runtime
Data point
controller
CEP
Component
-model
Elasticity
Auto scaling
group controller
Functional
capabilities
Non-functional
capabilities
...
...
Monitor.
Config.
Cloudcom 2014, 15 Dec, 2014,
Singapore
19
Stefan Nastic, Sanjin Sehic, Le-Duc Hung, Hong-Linh Truong, and Schahram
Dustdar (2014). Provisioning Software-defined IoT Cloud Systems. The 2nd
International Conference on Future Internet of Things and Cloud (FiCloud-2014),
August27-29, 2014, Barcelona, Spain.

20. People: incorporating humans into a
programming paradigm
Volunteers
Individual Team Professionals
Service-based Middleware
Monitoring Communication
Capability/Profile
Management
Provisioning/Negotiation/Execution API
Abstraction of Human-based Compute Units
ICU ICU SCU SCU
Program languages and programming models
Program
elements
Software
, Things
Compute
Units
program human actions
and dependencies
program incentive condition
and rewarding action
program result evaluation
method
Human-to-middleware
interfaces:
•visualization of collective tasks
•embedding of common forms
•mobile app
20
Cloudcom 2014, 15 Dec, 2014,
Singapore
SCU
Hong Linh Truong, Schahram Dustdar, Kamal
Bhattacharya: Programming Hybrid Services in the
Cloud. ICSOC 2012: 96-110 (Best Paper Award)

21. PART 2: TECHNIQUES FOR
PROGRAMMING AND
ENGINEERING ELASTICITY
FOR NATIVE CLOUD
SERVICES
Cloudcom 2014, 15 Dec,
2014, Singapore
21

22. Service
Developer
Infrastructure
Provider
Service
Owner
Service
Developer
Designing and
programming software-defined
elastic services
Automatic deployment
and configuration
Elasticity monitoring and
analysis
Elasticity Control
Service
Owner
Infrastructure
Provider
Service
Owner
Easy to
program
elasticity
requirements
Reduced time to market,
easy to reconfigure
+
Easy to understand
service’s elasticity
boundaries
+
Maintains service’s
performance while
reducing cost
Reduces
resources
overprovisioning
+
Elastic cloud service engineering
Cloudcom 2014, 15 Dec, 2014,
Singapore
22
Native elastic cloud service engineering
Several owners,
developers and
providers from
different
organizations

23. Elasticity
Metrics
Elasticity
Requirements
Elasticity capabilities
(e.g. scale IN/OUT)
So what need to be done? A simple
view
Cloudcom 2014, 15 Dec, 2014,
Singapore
23
Programming and deploying services

24. Fundamental building blocks for
the elasticity
 Conceptualizing and modeling elastic objects and
execution environments
 So we can manage diverse types of artifacts and their runtime
in a similar manner
 Defining and capturing elasticity primitive operations
associated with elastic objects and environments
 Recommending and Programming elastic objects
 a software-defined elastic system (SES) is built from elastic
objects
 Runtime deploying, control, monitoring and testing
techniques for elastic objects
Cloudcom 2014, 15 Dec, 2014,
Singapore
24

25. Need to model/capture elasticity
primitive operations
Cloudcom 2014, 15 Dec, 2014,
Singapore
25

26. Deploying, Control, Monitoring and
Testing
 Runtime configuration
 Complex services at multiple software stacks (VM,
OS container, ApplicationContainer, Web services,
etc)
 Interfaces with different low-level deployment techniques
 Different interactions between deploying and control
and monitoring components
 Control elasticity
 Using a high-level specification for specifying
elasticity requirements, constraints and strategies
 Based on SYBL/rSYBL ([CCGrid 2013, ICSOC 2013,
ICSOC 2014])
Cloudcom 2014, 15 Dec, 2014,
Singapore
26

27. Deploying, Control, Monitoring and
Testing
 Elasticity monitoring and analysis
 Utilize low-level metrics to build „Elasticity Space“
and analyze the elasticity based on such spaces
(based on MELA – [CloudCom 2013, CloudCom
2014, IJDBI])
 Monitoring/analysis at multiple levels level (single
unit, topology/group, and the whole service
 Testing elasticity
 Using clouds to test cloud applications as well as to
test elasticity properties of cloud applications
[ASE2013, IC2014]
Cloudcom 2014, 15 Dec, 2014,
Singapore
27

28. Software-defined
Elastic System
Programming
Tooling – Elasticity
Programming in
Cloud Systems
Elastic Service
Ecosystem and
Recommendation
Deployment
Deployment
Service
Test Generating
and Execution
Elastic Test
Service
deploy SDS/
service units
deploy elasticity
controller and monitor
Elasticity
Analysis
deploy
test cases
Elasticizing
Elasticity
Monitoring
and Analysis
Elasticity
Control
test
control
monitor
CoMoT
Core Services
Multi-Cloud
Environments
Service
Ecosystems
Service Artifact
Repository
Service units
CoMoT (1)
Cloudcom 2014, 15 Dec, 2014,
Singapore
28
Hong-Linh Truong et al., "CoMoT – A Platform-as-a-Service for Elasticity in the Cloud", IEEE International
Workshop on the Future of PaaS. Colocated with IC2E 2014

29. CoMoT (2)
 CoMoT is built atop:
 QUELLE, rSYBL, MELA, SALSA
 Work on multi-cloud environments
 Parts of complex applications are deployed in
different clouds
 http://tuwiendsg.github.io/
Cloudcom 2014, 15 Dec, 2014,
Singapore
29

30. QUELLE – QUantifying ELasticity
utiLity Engine
Cloudcom 2014, 15 Dec, 2014,
Singapore
30
Daniel Moldovan, Georgiana Copil, Hong-Linh Truong, Schahram Dustdar, QUELLE – a Framework for
Accelerating the Development of Elastic Systems, ESOCC 2014. September 2014

31. SALSA – runtime configuration
framework
https://github.com/tuwiendsg/SALSA
Cloudcom 2014, 15 Dec, 2014,
Singapore
31
Support cloud
services at data
centers and IoT
environments

32. rSYBL – Elasticity Control Engine
Cloudcom 2014, 15 Dec, 2014,
Singapore
32

33. MELA -- Elasticity Monitoring as a
Service
Cloudcom 2014, 15 Dec, 2014,
Singapore
33
Daniel Moldovan, Georgiana Copil, Hong-Linh Truong, Schahram Dustdar, MELA: Elasticity Analytics for Cloud Services,
International Journal of Big Data Intelligence, 2014

34. Toolsets and actions for elasticity
control, analytics and management
Cloudcom 2014, 15 Dec, 2014,
Singapore
34

35. Multi-cloud, multi-stack, complex
topologies configuration
Cloudcom 2014, 15 Dec, 2014,
Singapore
35
 Well-defined APIs for manipulating and provisioning objects
 Support different types of objects, e.g., VMs, OS containers,
services, service containers, IoT sensors, gateways,
Data center services Sensors
Come to see our paper at Section 2A , 15:30-17:30, Tue 16 Dec

36. High level elasticity control
#SYBL.CloudServiceLevel
Cons1: CONSTRAINT responseTime < 5 ms
Cons2: CONSTRAINT responseTime < 10 ms
WHEN nbOfUsers > 10000
Str1: STRATEGY CASE fulfilled(Cons1) OR
fulfilled(Cons2): minimize(cost)
#SYBL.ServiceUnitLevel
Str2: STRATEGY CASE ioCost < 3 Euro :
maximize( dataFreshness )
#SYBL.CodeRegionLevel
Cons4: CONSTRAINT dataAccuracy>90%
AND cost<4 Euro
Georgiana Copil, Daniel Moldovan, Hong-Linh Truong, Schahram Dustdar, "SYBL: an Extensible Language for Controlling
Elasticity in Cloud Applications", 13th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid),
May 14-16, 2013, Delft, Netherlands
Cloudcom 2014, 15 Dec, 2014,
Singapore
36

37. Configurations for multiclouds
Cloudcom 2014, 15 Dec, 2014,
Singapore
37
..
<ServiceElasticityPrimitives id="FCO"
ServiceProvider="Flexiant FCO">
<ElasticityPrimitive id="ScaleIn" name="Remove
VM" parameters=""/>
<ElasticityPrimitive id="ScaleOut" name="Create
new VM" parameters=""/>
<ElasticityPrimitive id="AllocateIP" name="Allocate
public IP" parameters="UUID"/>
<ElasticityPrimitive id="AttachDisk" name="Attach
NewDisk" parameters="UUID">
…
<PrimitiveDependency
dependencyType="AFTER_ENFORCEMENT"
primitiveID="Reboot"/>
…
MultipleEnforcementPlugins =
flexiant:at.ac.tuwien.dsg.rSybl.cloudInteractionUnit.enfor
cementPlugins.flexiant.EnforcementFlexiantAPI,
openstack:at.ac.tuwien.dsg.rSybl.cloudInteractionUnit.en
forcementPlugins.openstack.EnforcementOpenstackAPI
Configuraring low-level
Plug-ins to work with
multiple clouds
Configuraring
and capturing
elasticity
primitive
operations
associated with
service units

38. Mapping Services Structures to
Elasticity Metrics
Cloudcom 2014, 15 Dec, 2014,
Singapore
38
Elasticity directives
can be changed at
runtime

39. Multi-cloud example
M2MDaaS:
STRATEGY CASE avgBufferSize<5 :
minimize(cost)
CONSTRAINT avgBufferSize<50 CONSTRAINT responseTime<50 ms
STRATEGY CASE
responseTime<40ms AND
throughput<20ops/s : scalein
Load relationship
bufferSize requests
Cloudcom 2014, 15 Dec, 2014,
Singapore
39
Georgiana Copil, Daniel Moldovan, Hong-Linh Truong, Schahram Dustdar, "On Controlling Cloud Services
Elasticity in Heterogeneous Clouds", 6th Cloud Control Workshop, 7th IEEE/ACM International Conference on
Utility and Cloud, 2014

40. Elasticity Analytics for Cloud
Services
Moldovan D., G. Copil,Truong H.-L., Dustdar S. (2013). MELA:
Monitoring and Analyzing Elasticity of Cloud Service. CloudCom
2013
Elasticity space functions: to determine if a
service unit/service is in the “elasticity behavior”
Elasticity Pathway functions: to characterize the
elasticity behavior from a general/particular view
Elasticity Space
Cloudcom 2014, 15 Dec, 2014,
Singapore
40

41. Multi-level monitoring and analysis
Cloudcom 2014, 15 Dec, 2014,
Singapore
41

42. Multi-level monitoring and analysis
4
2
Elasticity Function for Demo
Apply different
elasticity
path/space
functions
Cloudcom 2014, 15 Dec, 2014,
Singapore
Cloud Service
Service
Topology
Service Unit
Code Region
Scopes in service structure
Common
functions/user-defined
functions
Serveral possible Elasticity Space
and Pathway functions
 for different types of service and
elasticity behaviors
Elastic test
frameworks
Benchmarks
Machine
learning

43. Examples of functions for Elasticity
Space and Pathway
Change point detection
algs
Alessio Gambi, Daniel Moldovan, Georgiana Copil, Hong Linh Truong,
Schahram Dustdar: On estimating actuation delays in elastic computing
systems. SEAMS 2013: 33-42
Elasticity Space Func
Elasticity Pathway Func
cost
Pathway 2 quality
cost
Pathway 1 quality
Cloudcom 2014, 15 Dec,
2014, Singapore
43

44. Elasticity Space Monitoring as a
Service
 Elasticity Signature Function Customization
 Uses simple composition rules (+,-,*,/, AVG, MIN/MAX, KEEP
LAST and CONCAT)
 Defines different composite metrics for different entities at each
level of the app structure
 Each composite metric is propagated to the higher-level parent.
 Elasticity Signature Boundary
 Different restrictions attached to different levels and entities
Cloudcom 2014, 15 Dec, 2014,
Singapore
44
Elasticity Function for Demo

45. Elasticity space and pathway analytics
Cloudcom 2014, 15 Dec, 2014,
Singapore
45
Daniel Moldovan, Georgiana Copil, Hong-Linh Truong, Schahram Dustdar, MELA: Elasticity Analytics for Cloud Services,
International Journal of Big Data Intelligence, 2014

46. Elasticity dependency analysis
 The elasticity of a service unit affects the elasticity of another unit.
How to characterize such cause-effect: elasticity dependency
 Modeling collective metrics evolution in relation to requirements
Cloudcom 2014, 15 Dec, 2014,
Singapore
46
Daniel Moldovan, Georgiana Copil, Hong-Linh Truong, Schahram Dustdar, On Analyzing Elasticity Relationships of Cloud
Services, 6th International Conference on Cloud Computing Technology and Science, 15-18 December 2014, Singapore
Come to see our paper at Section 4C , 10:30-12:00, Thu 18 Dec

47. Enable elasticity reconfiguration at
runtime
Cloudcom 2014, 15 Dec, 2014,
Singapore
47
Analysis detects problems
but predefined strategies do
not always work!
Changing elasticity
specifications at runtime
without stoping services

48. Elasticity behavior learning
Complex
Cloud
Service
Elastic
Cloud
Service
(running)
Deployed Elasticity
control
process
Elasticity Control
Processes
What would be
the elasticity
behavior?
Elasticity
requirements
Elasticity controller
Georgiana Copil, Demetris Trihinas, Hong-Linh Truong, Daniel Moldovan, George Pallis, Schahram Dustdar, Marios Dikaiakos.
"ADVISE - a Framework for Evaluating Cloud Service Elasticity Behavior" the 12th International Conference on Service
Oriented Computing. Paris, France, 3-6 November, 2014. (Best Paper Award)
Which elasticity controls are
the best for a given
situation?
Learning process
Control
Cloudcom 2014, 15 Dec, 2014,
Singapore
48

49. Part 3: Hand-on exercise
http://tuwiendsg.github.io/COMOT/elasticityDemo.html
https://t.co/eNY4rAAxXe
Cloudcom 2014, 15 Dec, 2014,
Singapore
49

50. Summary (1)
 Support principles of multi-dimensional elasticity
to achieve flexibility and dynamics in multi-cloud
environments for complex problems
 Coordinating elasticity across platforms need
concepts of elastic objects and fundamental
building blocks for engineering an end-to-end
elasticity for cloud services
 Need runtime elasticity techniques for dealing
with diverse types of services
Cloudcom 2014, 15 Dec, 2014,
Singapore
50

51. Summary (2)
 COMOT introduces concepts of elastic objects
and fundamental building blocks for engineering
an end-to-end elasticity for cloud services
 COMOT supports design, deployment, control,
monitoring and testing of elasticity in interwoven
engineering phases
 COMOT, SALSA, SYBL, and MELA provides ready-to-use
prototypes for domain-specific languages,
configuration, analytics and control services for
elasticity
 Tested in real-world scenarios and industrial settings
Cloudcom 2014, 15 Dec, 2014,
Singapore
51

52. Thanks for your attention!
Hong-Linh Truong
Distributed Systems Group
Vienna University of Technology
truong@dsg.tuwien.ac.at
dsg.tuwien.ac.at/research/viecom
https://twitter.com/dsg_tuwien
https://t.co/eNY4rAAxXe
Cloudcom 2014, 15 Dec, 2014,
Singapore
52