Description: Presentation at the International Industry-Academia Workshop on Cloud Reliability and Resilience. 7-8 November 2016, Berlin, Germany. Organized by EIT Digital and Huawei GRC, Germany. Twitter: @CloudRR2016

1. DICE Horizon 2020 Project
Grant Agreement no. 644869
http://www.dice-h2020.eu Funded by the Horizon 2020
Framework Programme of the European Union
Cloud Based Fault Injection for
Anomaly Detection
International Industry-Academia Workshop on
Cloud Reliability and Resilience
Berlin, 7-8th November 2016
Craig Sheridan
flexiOPS Limited
Managing Director

2. DICE RIA - Overview
Overview
o Cloud Evolution
o The DICE Horizon 2020 Project
o Technical Overview
o Problem, objectives & positioning
o Demonstrators
o Fault Injection Tool
o H2020
2©DICE

3. Evolution of the Cloud

4. DICE Horizon 2020 Project
Grant Agreement no. 644869
http://www.dice-h2020.eu Funded by the Horizon 2020
Framework Programme of the European Union
DICE: Developing Data-Intensive
Cloud Applications with Iterative
Quality Enhancements
International Industry-Academia Workshop on
Cloud Reliability and Resilience
Berlin, 7-8th November 2016
Craig Sheridan
flexiOPS Limited
Managing Director

5. DICE RIA - Overview
DICE Mission and Partners
 ICT 9 Call/2014 – Software engineering
 9 partners (Academia & SMEs), 7 EU countries
5©DICE
Mission: support SMEs in developing high-quality
cloud-based data-intensive applications (DIAs)
(IEAT)
(IMP)
(PMI)
(ZAR) (NETF)
(XLAB)
(ATC)
(FLEXI)(PRO)

6. Big Data Technologies
Cloud Testbed
`
DICE RIA - Overview
DICE Framework (IDE-centric)
6©DICE
DICE IDE
Profile
Plugins
Quality Analysis
UML-
Based
MDE
Methodology
Feedback &
Iterative
Enhancement
Data Intensive Application (DIA)
Continuous Delivery & Testing
WP6 - Demonstrators
WP4
WP3
WP2
WP5
WP1

7. DICE RIA - Overview
The Rapid Growth of Big Data
7©DICE
o Software market rapidly shifting to Big data
 27% compound annual growth rate through 2017 (IDC)
 Popular technologies such as Spark, Hadoop, and NoSQL
boost Big Data adoption and revenues from new services
Business issue: 65% of Big data projects still fail (CapGemini)
Source: IDC Source: Wikibon

8. DICE RIA - Overview
What problems EU SMEs face?
8©DICE
Traditional market:
Legacy software systems
Customers with legacy
data now ask for Big
Data technologies
Growth in sight, but …
Learning curves
Initial prototyping
Risk of failure
(+ others…)
Fast-paced market

9. DICE RIA - Overview
How to support DIA development?
9©DICE
Characterize Data Properties
Locatio
n
Velocity
Volume
Variety
Privacy
Big Data
Technologies
NoSQL
Hadoop
Spark
Storm
Development
Methods & Tools
UML
Delivery
DevOps
Cloud
Integration
QA

10. o Reliability
o Efficiency
o Correctness
DICE RIA - Overview
What do we mean by Quality?
10
 Availability
 Fault-tolerance
 Performance
 Costs
©DICE
 Privacy
 Temporal metrics

11. DICE RIA - Overview
DICE Objectives & Positioning
11©DICE
DICE delivers the first
quality-driven
framework for DIAs:
 UML profile for DIAs
 UML-driven agile
delivery via DICE IDE
 Ecosystem of quality
prediction and testing
tools for Big data
technologies
 Iterative cycles of
refinements through
analysis of test data

12. DICE RIA - Overview
Demonstrators
12©DICE
News&Media
Market
e-Goverment
Market
Maritime
Sector
Tax Fraud Detection
Application
Posidonia OperationsNews Asset

13. DICE advantages
o Tackling skill shortage & steep learning curves in quality-
driven development and Big Data technologies through
open source development tools, models, and methods.
o Shorten time to market for data-intensive apps that
meet quality requirements, reduce costs for software
vendors, increase value for end users.
o Reduce number & severity of quality related incidents by
iteratively learning application runtime behavior, feeding
back the information to the developers.
DICE RIA - Overview 13
©DIC

14. Fault Injection Tool
DICE RIA - Overview 14
©DIC

15. Big Data Technologies
Cloud (Priv/Pub)
`
DICE Framework
15©DICE
DICE IDE
Profile
Plugins
Sim Ver Opt
DPIM
DTSM
DDSM TOSCAMethodology
Deploy Config Test
M
o
n
Anomaly
Trace
Iter. Enh.
Data Intensive Application (DIA)
Cont.Int. Fault Inj.
WP4
WP3
WP2
WP5
WP1 WP6 - Demonstrators

16. Motivation
o Robust testing, showing where to harden
application before it reaches a commercial
environment
o Allows the tool user to generate faults either at
VM or cloud level. Lets cloud platform owners
test resiliency of a cloud installation as an
application target.
o Thus allowing for the mitigation of risk in advance
of a cloud based deployment.
DICE RIA - Overview 16
©DIC

17. Innovation
o ChaosMonkey- only accesses AWS & terminates VMs. FIT
generates various cloud agnostic faults at VM & Cloud.
o Cocoma- developed but not supported, suffers from high
resource usage, complex config & limited extensibility.
o Range of functions, greater flexibility to generate
multiple faults
o Lightweight and only installs required tools and
components on target VMs.
o Future extensibility in mind considering the needs and
challenges of cloud service providers such as scalability
and resiliency of the cloud consumption marketplace.
DICE RIA - Overview 17

18. Design
DICE RIA - Overview 18
©DIC

19. Supported Faults
DICE RIA - Overview 19
©DIC

20. Usage example result
DICE RIA - Overview 20
©DIC
o --stressmem 2 2048m ubuntu@109.231.126.101 -
no home/ubuntu/SSHKEYS/VMkey.key

21. Conclusion and future
o Aimed at developers using quality driven DevOps
approach to test apps under conditions of failing
infrastructure or misbehaving services.
o Difficulties and possibilities of extensibility by external
users and investigating limitations.
o consider different topologies, operating systems and
vendor agnostic cloud provider infrastructure as well as
evaluating the overhead of operation.
o Containerised environments will be a future target to
help understand effect on microservices when injecting
faults to the underlying host.
DICE RIA - Overview 21
©DIC

22. DICE RIA - Overview
Questions?
22©DICE
Thanks!