Future of Testing 2018 Webinar with Bloor, February 1, 2018

1. 1
10 Steps to State-of-the-Art
Software Testing
Dr. John Bates, CEO

2. 2
Testing Still in the Dark Ages

3. 3
10 Steps to the State of the Art
1. Test for business assurance.
2. Prioritize UX in testing.
3. Adapt testing to new software
architectures.
4. Streamline testing to keep up
with continuous delivery.
5. Use AI for intelligent test
creation.
6. De-risk outcomes using
predictive analytics.
7. Model digital test twins.
8. Supercharge humans via
scriptless testing.
9. Execute UX monitoring for
continuous testing.
10. Perform smart testing as a
service.

4. 4
#1 Test for Business Assurance
Quality Assurance Business Assurance
• QA transitions to a critical business function
• Requires new technologies that:
• Understand the business goals.
• Automatically create and orchestrate tests to ensure goals are met.
• Use continuous testing to monitor business goals.

5. 5
Lives Saved
9456
Cerner — Lives Saved

6. 6
U.S. Missile Defense Agency
— Lives Protected

7. 7
NASA Orion — Mission Success

8. 8
Walmart — Outperforming Amazon

9. 9
Pizza Hut — Topping Delight

10. 10
#2 Prioritize UX in Testing
18%
Actually meeting
customer
expectations
User Experience Gap
Teams feel they are
meeting their
test objectives
86%
App still used
after 1
month
Teams say they have
no idea how to test
the real UX
4%
63%

11. 11
Test UX From the User Perspective

12. 12
Test UX From the User Perspective

13. 13
#3 Adapt Testing to New Software Architectures
 Digital experience
 DevOps
 Mobile and IoT
 Microservices Monolithic
architecture
API-driven microservices
architecture:
cloud(s) and edge
About the code + About UX & APIs

14. 14
Create Fusion Test Automation Engines
APIs
Objects
• Platform-agnostic
• User-centric, API, and
objects
• Functional,
performance, UX …

15. 15
#4 Streamline Testing to Keep Up With CD
Develop Build Test Deploy Release
Continuous Delivery
91%
Continuous Delivery Gap
Developers agree that user
expectations for innovation
and quality have increased
40%
Developers confess to
releasing apps without
testing
89%
QA teams say
that they can’t
keep up

16. 16
Focus Test Automation Beyond Only Test Execution
Define
Test
Objectives
Define
Test
Cases
Create &
Maintain
Test
Scripts
Define
Test Runs
Setup
Test Env
Review
Test
Results
Feedback
Execute
Tests
(auto)

17. 17
Let’s Automate the Rest of the Testing Process …
Utilize AI and big data analytics
Define
Test
Objectives
Define
Test
Cases
Create &
Maintain
Test
Scripts
Define
Test Runs
Setup
Test Env
Review
Test
Results
Feedback
Execute
Tests
(auto)

18. 18
#5 Automate Test Creation Via Machine Learning
Common user journeys
(from usage logs).
Historical patterns and
their relationship to
customer satisfaction.
Prioritized test cases.
Modules where code
has churned and
developers that worked
on them.
Patterns of defects seen
in the past and what
modules these were
seen in.
AI builds test cases for
prioritized & common
user journeys, then
explores permutations
around them to hunt
bugs.
AI explores paths a
human would never
think of and
intelligently refines
exploration to
maximize coverage.
AI prioritizes churned
code modules and
create test cases to test
quirks of particular
developers and
historical bug patterns.
AI predicts how the
characteristics of this
release will impact
customer satisfaction.

19. 19
#6 De-Risk Outcomes Using Predictive Analytics
Dashboard
Reporting
What-if
analysis

20. 20
#7 Model Digital Test Twins

21. 21
#8 Supercharge Humans Via Scriptless Testing
 Make test automation accessible to nonprogrammers.
 Enable domain experts to focus on business success.
 Empower a human to be 10–100x more productive.

22. 22
#9 Execute UX Monitoring for Continuous Testing
Fusion engines
(Func., perf., UX)
Model
Eggplant AI
Autogenerated
scripts
App
(live)
App
(in development)
Analytics
(incl. release readiness dashboard)
Synthetic
monitoring
Real user
monitoring
Progress to PBOs
NumberofUsers
Progress to PBOs
NumberofUsers
$$$

23. 23
Eggplant
Automation
Cloud
#10 Perform Smart Testing as a Service
Vertical
test
solutions
Eggplant
Analytics
Eggplant
AI Eggplant
Functional
Eggplant
Performance
Jenkins
HP ALM
JIFA
Selenium
JUnit
UFT
CI/CD
environment

24. 24
10 Steps to the State of the Art
1. Test for business assurance.
2. Prioritize UX in testing.
3. Adapt testing to new software
architectures.
4. Streamline testing to keep up
with continuous delivery.
5. Use AI for intelligent test
creation.
6. De-risk outcomes using
predictive analytics.
7. Model digital test twins.
8. Supercharge humans via
scriptless testing.
9. Execute UX monitoring for
continuous testing.
10. Perform smart testing as a
service.