With a constant flow of new devices and operating systems, and ever-changing user conditions, knowing exactly which ones to test can be a challenge for web and mobile app teams. That’s why we created FACTORS. Now celebrating its 10th edition, FACTORS is the data-driven go-to resource for app developers, testers and owners for building a cross-platform test strategy. Every issue includes a current benchmark of devices, web browsers and user conditions – by geography – to aid in your pursuit of digital excellence. Join us for this 30-minute webinar with Perfecto’s Eran Kinsbruner, the creator of FACTORS, to discover the best ways to incorporate this data-driven approach to your 2019 continuous testing & DevOps strategy. Learn how to: -Develop a test coverage plan that reflects your actual users -Factor in regional differences with worldwide data insights from 20 countries -Align cross-platform testing into the development cycle – what to test on at what stage -Plan your 2019 lab by testing on the relevant devices, operating systems, and browsers and keep ahead of the market!

1. End the Guesswork: How to
Test the Right Devices
,Platforms and User
Conditions

2. HOUSEKEEPING
• Please participate in our live poll
• Ask us questions in the Q&A panel
• We will send you the recording and slides
11/10/2017 2© 2015, Perfecto Mobile Ltd. All Rights Reserved.

3. AGENDA
• Market Overview & Continuous Testing
• How to develop a test coverage plan that reflects your actual users
• Align cross-platform testing into the development cycle – what to
test on at what stage
• Plan your 2019 lab by testing on the relevant devices, operating
systems, and browsers and keep ahead of the market!
• Bonus: ML and Future of CT
Eran Kinsbruner
Dir., Lead SW
Evangelist and
Author, Perfecto
Twitter: @ek121268 (https://twitter.com/ek121268)
Blog: http://continuoustesting.blog
LinkedIn: https://www.linkedin.com/in/eran-kinsbruner-4b47a81/

4. Continuous Testing for Digital Has Never Been
More Complex
5/24/2018 4© 2018, Perfecto Mobile Ltd. All Rights Reserved.
Augmented Reality
Virtual Reality
IOT
Artificial Intelligence
Mobile
Progressive Web
Continuous
Testing for
Digital

5. There Are Patterns for “Unstable” Test Automation
80% of issues have a pattern52% success rate
10% of devices,
causing 80% of lab
issues
Lab
25%
Orches
tration
25%
Scripts
& FW
50%
FAILURE REASON
Objects Codding Time Other
Scripts & FW issues
Device in use
No Device
Orchestration issues
Networking Stability Lock
Other
Lab issues
What’s
wrong
With my
Scripts
What’s wrong
With my Lab
What’s wrong
With my
Executions

6. Mobile & Web Market
Overview
Coverage Methodology

7. Mobile
Landscape
28.2%
21.3%
17.9%
21.5%
7.6%
3.9%

8. Web
Landscape

9. What’s new?
• Refreshed Country Data
• Q4 2018 & Q1 2019 Mobile Calendars
• Web, RWD Coverage Refresh
• Market Overview Insights (Devices, PWA, Big Data)
Perfecto’s Factors Magazine - 10th Edition
Where to get it from: https://info.perfectomobile.com/v10Factors.html

10. Methodology

11. U.S & EU5 Recommended Coverage Q4 2018

12. Q42018 & Q12019 Calendars

13. Coverage Throughout the DevOps Pipeline (Process)
12/20/2018 13© 2015, Perfecto Mobile Ltd. All Rights Reserved.
PLAN CODE BUILD TEST RUN
New Functionality
(Innovation)
Users
User experience

14. Future Of Testing

15. Key ML Use Cases In Test Automation
Recognize objects
Transcribe speech – Accessibility
Make quality related decisions based on data
Identify Trends and/or Patterns
Security use cases – Identify signatures e.g.
12/20/2018 15© 2015, Perfecto Mobile Ltd. All Rights Reserved.

16. Training AI the Testing
Lingo
12/20/2018 16 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
PopupApp not installed
Elementnotfound
App didn’t launch
Capability not supported
App Crashed
Keyboard
Device in Use
Trust

17. Training the Testing AI
Engine – Image
Recognition
• Stabilize Execution By Eliminating Noise
• Pop ups
• Home screen
• Keyboard open
• Enhanced Reporting and Analysis
12/20/2018 17 © 2015, Perfecto Mobile Ltd. All Rights Reserved.

18. Mindset and Workflow Changes in Test Automation
12/20/2018 18© 2015, Perfecto Mobile Ltd. All Rights Reserved.
# Category
1 Test Authoring
2 Test Maintenance
3 Test Execution
4 Test Analysis
5 Tools Maturity
6 Available Integrations
7 Req. Test/Dev Skillset
8 Testing Env.
9 Testing Types
10 Supported App Types
Test Engineers/Developers Manual Testers (Developers?)
Traditional Test Automation
Define Manual Flows, BDD Style, etc.
Test Code & Reusable Functions (Java, JS,
etc.)
Define Objects, POM, use Object Spy
Structured Page Based Test Steps
Define Visual Validations and Assertions
Longer Time To Develop, Complex
Changes required pro-actively, SCM
Supported
Configure Env. (TestNG Data Provider),
Execute Locally/CI/Cloud
Flexible, OSS, Commercial
High, Including Guidelines, Doc’s
Plenty, OSS, Defect Management, etc.
Medium-High
IDE
API, Load, Functional
All Types
ML Based Test Automation
Record Test Flows (No Coding – in most cases)
Groups Reusable Functions
Objects Generated On-The-Fly (transparent to user)
Flow Based Scenarios
Part of Test Authoring Recording/Authoring
Shorter Time, Advanced Capabilities
Self-Healing/Correction Automatically handled/Object
Scoring/Local SCM Controlled
Configure Env., Execution Management Built-In,
Locally/CI/Cloud
Relies on ML/AI Tool Vendor in Most Cases
Emerging Technology, Web More Advanced Than
Mobile
Most capabilities built-in, some exists
Low-Medium
ML UI
Mostly Functional (E2E) & API
Mostly Web

19. Q & A

20. Thank You