While in contemporary research the elasticity is mostly interpreted as the capability of scaling in and out of machine-based computing resources within a single cloud, we argue that the elasticity can be performed in multiple dimensions, such as resource, cost, and quality. Furthermore, the elasticity can also be performed in hybrid machine- and human-based computing systems. In this talk, first we will discuss principles of such elastic processes and their research challenges. Second, we will present some initial results of techniques that can be used in the development of elastic processes, such as composable cost evaluation, contract compatibility evaluation, and quality of data evaluation for workflows using both software and human capabilities. Finally, we will outline challenges when bringing these techniques to hybrid systems of software- and human-based computing elements.

1. Presented at Australian Symposium on Services Innovation
November 21st and 22nd, 2011, University of Wollongong
Principles of Elastic Processes on
Clouds and Some Enabling Techniques
Hong-Linh Truong
Distributed Systems Group, Vienna University of
Technology
truong@infosys.tuwien.ac.at
http://www.infosys.tuwien.ac.at/Staff/truong
Joint work with: Schahram Dustdar, Frank Leymann, Michael Reiter, Tran
Vu Pham, Quang Hieu Vu, Yike Guo, Benjamin Satzger, Uwe
Breitenbuecher, Dimka Karastoyanova
ASIS' 11, Wollongong, Nov 22, 2011 1

2. Outline
 Principles of elastic processes
 Some enabling techniques for elastic processes
in hybrid systems
 Monitoring and analysis of non-functional parameters
ASSI' 11, Wollongong, Nov 22, 2011 2

3. Data-as-a-service and computation-
as-a-service
 Multiple realtime data sources – data services
 Smart environments
 Earth monitoring, satellite images
 Social media Web information
 Personal medical information
 Data processing algorithms – computing services
 Data conversion, data enrichment, data extraction, data
mining, domain-specific computational analysis (e.g.,
CO2 calculation), etc
 Algorithms can be wrapped into processing
elements/workflow activities
ASSI' 11, Wollongong, Nov 22, 2011 3

4. Data intensive processes
 Influence factors for the quality of processed
results
 Data rates, quality of data sources, underlying
computing systems, etc.
 The quality of the output is varying
 Multiple concurrent consumers
 Dynamic requirements for SLA, quality of outputs, etc.
Data processing service provider: my process must
be elastic!
 to optimize resource usage, prices, etc., but must meet
consumer's requirements
ASSI' 11, Wollongong, Nov 22, 2011 4

5. Example of a business/e-science
process Influence Quality/cost influence?
factors:
performance
algo2 quality of data Outputs:
results,
algo4 algo5 accuracy,
algo1
performance
algo3 algo6 etc.
Quality of data?
algo5 algo8 =?
algo4
Reqs:
DaaS algo7 Consumer 1,
Consumer 2
Resources, quality and cost? Consumer n
Resources in multiple clouds
ASSI' 11, Wollongong, Nov 22, 2011 5

6. Elasticity in computing
„Elastic computing is the use of computer resources
which vary dynamically to meet a variable workload” –
http://en.wikipedia.org/wiki/Elastic_computing
„Clustering elasticity is the ease of adding or removing
nodes from the distributed data store” –
http://en.wikipedia.org/wiki/Elasticity_(data_store)
„What elasticity means to cloud users is that they should
design their applications to scale their resource
requirements up and down whenever possible.“, David
Chiu – http://xrds.acm.org/article.cfm?aid=1734162
ASSI' 11, Wollongong, Nov 22, 2011 6

7. Elasticity in computing – a broad
view in multi-cloud environments
 Elastic demands from consumers
 Multiple outputs with different price and quality (output
elasticity)
 Elastic data inputs, e.g., deal with opportunistic data
 Elastic pricing and quality models associated resources
 The service/process itself is elastic in terms of economic
theory
 The service/process structure is elastic in terms of physics
ASSI' 11, Wollongong, Nov 22, 2011 7

8. Multiple elasticity dimensions
Resource elasticity:
software/human-based Non-functional parameters:
computing elements, multiple performance, quality of data,
clouds service availability, human
trust, ...
Pricing/Rewarding/Incentive
elasticity
Elasticity
ASSI' 11, Wollongong, Nov 22, 2011 8

9. Conceptual architecture of elastic
process environment
Schahram Dustdar, Yike Guo, Benjamin Satzger, Hong Linh Truong: Principles of Elastic Processes. IEEE Internet Computing 15(5): 66-71 (2011)
ASSI' 11, Wollongong, Nov 22, 2011 9

10. Elastic process – operation
principles
 Operation principles
 Monitor, manage and describe dynamic properties
 Dynamically refine process functions based on
quality
 Determine costs based on multiple resource
price/reward/incentive models
 Provide elasticity across multiple cloud providers
 An EP can deal with multiple service objectives
 N concurrent consumers, access markets of M
providers, with K different requirements: K <= N
ASSI' 11, Wollongong, Nov 22, 2011 10

11. Elastic process – modeling
principles
 Overlaying EPs, functional composition, and
dynamic property composition
 Modeling principles
 EP's function as a static property
 EP's results based on requirements concerning
price/reward/incentive and quality
 modeled as a set of constraints influencing resource
elasticity
 Communication can be based on SOAP/REST
 Refinement and composition of resource, price and
quality at multiple levels:
 activities, fragments, or the whole EP
ASSI' 11, Wollongong, Nov 22, 2011 11

12. The Vienna Elastic Computing Model
E-science Application Business Application
 Service modeling and
Elastic Service interface
 Alignment techniques
Elastic Alignment
 Virtualization techniques
IT Elastic Process  Monitoring and analysis
techniques
Elastic Runtime Management  Non-functional parameters
characterization
IT Elastic Service  Programming models
 Governance and service
Virtualization evolution
Software-based Human-based
Computing Element Computing Element  Heavy crowded in cloud computing and
social computing
(SE) (HE)  Ready to use but in isolation (single cloud
and single type of computing elements)
ASSI' 11, Wollongong, Nov 22, 2011 12

13. Clouds of hybrid systems
Software- and human-based
resource virtualization
 Middleware
 Modeling and
alignment
 Evolution
 Programming
languages
 Refinement and
adaptation
 NFP Characterization,
Monitoring and
Analysis
 Pricing/Rewarding/Ince
ntive mechanisms
 Governance and
compliance
ASSI' 11, Wollongong, Nov 22, 2011 13

14. How to determine the cost model of
a complex process?

Coarse- and fine-grainted cost models of clouds at different layers:
 Too coarse-grained (networks, storages, machines) or too fine-
grained (IO calls)
 Application-, software-, data-, human-specific cost/performance models
 Cost models for individual parts (workflow, MPI, OpenMP, etc.)
 Elastic cost models associated with resources and services in multiple
clouds
Elastic Process Part Part Part
A B C
Our approach
cost/performance cost/performance cost/performance
model i model j model k
ASSI' 11, Wollongong, Nov 22, 2011 14

15. How to monitor software based
computing elements across clouds?
Tran Vu Pham, Hong-Linh Truong, Schahram Dustdar "Elastic High Performance
Applications - A Composition Framework", The 2011 Asia-Pacific Services
Computing Conference (IEEE APSCC 2011), (c) IEEE Computer Society,
December 12 - 15, 2011, Jeju, Korea
Elastic high performance
applications on multiple
clouds
Hong Linh Truong, Schahram Dustdar: Composable cost estimation and monitoring for computational applications in cloud computing environments. Procedia CS 1(1): 2175-2184 (2010)
ASSI' 11, Wollongong, Nov 22, 2011 15

16. Are current “software service techniques”
suitable for DaaS/data marketplaces
Elastic dynamic properties associated with:
Data marketplaces and DaaS
Service APIs, data APIs and data assets
- Quang Hieu Vu, Tran-Vu Pham, Hong-Linh Truong, Schahram Dustdar, Rasool Asal,
DEMODS: A Description Model for Data-as-a-Service. AINA 2012. To appear.
- Hong-Linh Truong, Schahram Dustdar "On Analyzing and Specifying Concerns for Data as a
Service", The 2009 Asia-Pacific Services Computing Conference (IEEE APSCC 2009), (c) IEEE
Computer Society, December 7-11, 2009, Biopolis, Singapore.
- Marco Comerio, Hong-Linh Truong, Flavio De Paoli, Schahram Dustdar, " Evaluating Contract
Compatibility for Service Composition in The SeCO2 Framework ", The 9th International
Conference on Service Oriented Computing (ICSOC 2009), (c) Springer-Verlag, November 24 -
27, 2009, Stockholm, Sweden.
ASSI' 11, Wollongong, Nov 22, 2011 16

17. How to monitor quality of data in an
elastic way?
Pull, pass-by-references Pull, pass-by-values
Push, pass-by-values
Hong Linh Truong, Schahram Dustdar: On
Evaluating and Publishing Data Concerns for Data
as a Service. APSCC 2010: 363-370
ASSI' 11, Wollongong, Nov 22, 2011 17

18. How to deal with subjective and
objective evaluation in long
running processes?
 Our approach
 Different QoD
measurements
 Human and software
tasks
ASSI' 11, Wollongong, Nov 22, 2011 18

19. Experiments: cross-layered
monitoring in multiple clouds
Online analysis Estimation
Cloud provider bill
ASSI' 11, Wollongong, Nov 22, 2011 19 19

20. Experiments: evaluating and
publishing QoD with data resources
ASSI' 11, Wollongong, Nov 22, 2011 20

21. Experiments: evaluating quality of
data in workflows
Michael Reiter, Uwe Breitenbuecher, Schahram Dustdar, Dimka Karastoyanova, Frank Leymann, Hong-Linh Truong, A Novel Framework for Monitoring and Analyzing Quality of Data in
Simulation Workflows, (c)IEEE Computer Society, The 7th IEEE International Conference on e-Science, 5-8 December, 2011, Stockholm, Sweden
ASSI' 11, Wollongong, Nov 22, 2011 21

22. Open questions for enabling
techniques in hybrid systems
 Software, data and human integration on multiple clouds
 Monitor elastic processes in hybrid systems
 Provisioning on-the-fly monitoring tools for complex cost models from
different layers
 Monitoring human-based computing elements
 Check contract compatibility and compliance – no static
 Evaluating dynamic contract of data, software-based services, and
human-based computing elements in the same process
 Evaluate quality of data in complex workflows
 Modeling hybrid processes of human and software for evaluating quality
of data in data intensive workflows
 Utilizing quality of data in supporting the elasticity of cloud service and
infrastructure provisioning
ASSI' 11, Wollongong, Nov 22, 2011 22

23. Summary
 Elastic computing is needed in different aspects
 Scaling software, services, and people in the same
application
 Not just “resource elasticity”
 Resource, price/reward/incentive and quality
 Hybrid systems of software-based and human-based
computing elements
 Multiple clouds
 Several open research questions for realizing
elastic processes of hybrid computing elements
ASSI' 11, Wollongong, Nov 22, 2011 23

24. Thanks for your attention!
Hong-Linh Truong
Distributed Systems Group
Vienna University of Technology
Austria
truong@infosys.tuwien.ac.at
http://www.infosys.tuwien.ac.at
ASSI' 11, Wollongong, Nov 22, 2011 24