In this webinar, Adam will explain the benefits and restrictions that are encountered when working with Big Data systems in a modern agile development approach. He will go on to present some of the approaches, both in automation and in their management of testing activities, that his team has successfully adopted in tackling the big data testing challenge.
Tell me more - http://testhuddle.com/resource/big-data-a-new-testing-challenge/
4. @esconfs|#esconfs
Big Data – a Niche Market?
Demand for application development skills in big data
positions increased 396% in the first 6 months of 2015
as compared to the previous year
Source: Wanted analytics/Forbes – figures based on jobs quoting ‘Big Data’
5. @esconfs|#esconfs
Big Data is Coming
The Internet of Things is the term used to describe
network connectivity and communication between
machines and devices
6. @esconfs|#esconfs
Big Data is Coming
Gartner, Inc. forecasts that 4.9 billion connected
things will be in use in 2015, up 30 percent from 2014,
and will reach 25 billion by 2020 Source: Gartner (November 2014)
These are all connecting to share data
7. @esconfs|#esconfs
Big Data is Coming
There is an expectation in modern devices and systems of :-
• Connectivity with other systems
• Data collection
• Exposure to Analytic capability
Business
Asset
Insight
Business
Data
8. @esconfs|#esconfs
Big Data is Coming
Connected
devices
Activity data
capture
Data
collecting
monitors
Storage
Technology
Analytical
Technology
Management
and Control
Systems
Inevitably we’ll
see an increase
in….
…and the need
to test them.
10. @esconfs|#esconfs
Too Big to…..?
Back Up?
Manipulate?
Comprehend?
Count the Individual?
No… but too big to test every individual
No… but too big to manipulate the whole dataset
No… but too big to duplicate production for testing
No… but too big to consider scale at all levels at
once
14. @esconfs|#esconfs
Non-Functional?
What we may previously have described as ‘non-
Functional’ characteristics are the cornerstone of Big
Data technologies
• Scalability is a function of a Big Data system
• Performance is a function of a Big Data system
15. @esconfs|#esconfs
Non-Functional?
User stories/requirements are often focussed around “non-
functional” characteristics
“As a query user I want my lookup queries to run in 50% of the
time as they do at present.”
“As a cluster administrator I want to store over 1 million data
files with no loss of import performance”
We can’t leave testing of performance and scalability until
‘later’. Our everyday testing must focus on these.