App testing doesn't work if you don't use the mobile devices and platforms your customers are using.
We analyzed 360,000 app users and 4,000 device configurations to get the answers you need to test with confidence.
In this presentation, you'll learn:
» The top mobile device platforms across major geographies
» How to decide what mobile devices to develop and test on
» How to catch preventable defects that emulators miss
2. Answering the most common question:
“What should I test on to ensure
efficient market coverage?”
3. v
Mobile Test Coverage Index – Q1 2015
Covers (U.S & EU5): Devices, OS, Screen Sizes, Hardware
Thorough
Market
Analysis
360,000
Real End
User Data
4,000+
Proprietary
Device Data
6. Perfecto mobile
at a glance
The world leader in mobile application continuous
quality helping the world’s biggest brands to
deliver better apps faster
“Perfecto Mobile currently has the
the strongest 3rd party position in
the market” [ Thomas Murphy]
7. Why is it Important?
Keeps crashing my phone. Every time I use
Skype out to call regular phones on my
phone (Nexus 4) crashes and restarts. This
is pretty annoying!
Daniel Waisberg September 27,2013
Broken. No longer works on my
galaxy nexus.
Joshua Smith September 24,2014
Login Issues has been 3 days and
still can’t login to the mobile
app…was able to login to the
site just fine!
Eugene Vygocher September 24,2014
9. It’s Fundamental all around
For the Business it’s
market coverage
• Great end user experience
• Build requirement for IT
• Optimize user engagement & rating
For the Dev/QA it’s
test coverage
• Optimize process
• Discover bugs early, mitigate risk
• Continuously test on what matters
11. What does it take to cover the REAL
end user experience?
v
Device & Platform
Model
OS Version
Screen Size
Form Factors v
Environment
Location
Network
v
Conditions
Signal Strength
Load
Others
12. The
Index
Test Coverage Optimization Process
1. Coverage 2. Lab Sizing
Market Me
My Space
Popular & emerging Analytics
Industry trends & insights
Teams Projects
SDLC
Cycle
Requirements
13. Coverage Target
30% recommended device coverage
driven by market share and device
characteristics
30% and in addition market
leading devices, OS versions
etc.
50% and in addition more legacy &
new device/OS coverage
80%50%30%
10-16
Devices
17-25
Devices
26-32
Devices
14. The Discovery Process
v
Analytics Data Exists?
Yes
- Web Traffic?
- App Stores?
No
v
BU Requirements/ Weights?
APP TYPE
- B2C/B2E/Web/RWD?
- Web, Native, Hybrid?
BYOD vs. Market
Special Prioritizations?
- Demographics
- Platforms
- Geography
- Devices
v
Process & Maturity Related (Lab Size)?
- Agile/Waterfall
- Release cadence
- Automation Coverage
v
Customize Consideration Weights
- Analytics (Me)
- My Vertical
- Market
17. April OCTOBERJUNE JULY AUGUST SEPTEMBER
Sony Xperia
Z4 Tablet
New iPhones & iPads
Samsung Note 5
Motorola Moto X (New)
iOS 9 Beta
Windows 10
for Phones
DEVICES
PLATFORM
Mobile Market Calendar 2015
Android M ?
MAY
Apple Watch Release
Samsung Galaxy S6
Samsung Galaxy S6 Edge
LG G4
Nokia Lumia 640 XL
ZTE Blade S6 Plus
Huawei P8
Lenovo A6000 Plus
New trending devices,
First Samsung’s to run
Android L OTS
You can already start
Beta testing in June!
Seek for Emulators of
iPhone 6S and more close
to the release date!
Seek for early Beta
through Android Studio
Perfecto Mobile’s Q2
Index Release!!
23. THE MOBILE TEST COVERAGE INDEX
MAKE SURE YOUR APP WORKS
ON THE RIGHT DEVICES
Read the Mobile
Test Coverage Index
24. Mobile Test Coverage
Index – Q1 2015
OEM Insights
Apple rules the U.S market while Samsung the EU5
More Android OEM’s gain market share
OnePlus, ZTE, Alcatel, Huawei, Asus
Windows Phone twice stronger in EU5 than in U.S
25. Q1 2015 – U.S Index
Supports:
• Sufficient Device Mix
• Legacy, Reference, New
• Allows sufficient OS coverage
• Support mix of Screen Sizes
• Support mix of hardware’s
27. Summary
Free download The Q1 Mobile Test Coverage Index
Adopt Me, My Space & Market strategy
Note: Mobile Test Coverage is more than Devices & OS’s
Continuously evaluate your mobile quality lab
Apple holds almost as must share in the U.S as Samsung in EU5
Android device fragmentation is growing with more vendors showing growing adoption – OnePlus, ZTE, Huawei, Alcatel
Windows Phone is twice stronger in EU5 than in the U.S
HTC One M7 – 2.5 years old, can be considered legacy and running Android 4.x but getting this month Android L
2 Samsung generations, 3rd (S6) coming up in next index
Various Screen size/HW for RWD testing, market and vertical coverage mix
2 latest iPad tablet devices are MUST