overview of mobile app quality. based on learnings from 10000 app testers across thousands of apps and analytics on 100M appstore reviews.

1. app quality
overview

3. who am i? tester

4. eng director/developer
Analytics
1,000,000+ apps
100,000,000+ reviews
Crowd Testing
1,000+ apps
100,000+ Testers

6. app startup

7. app-specific issues
●
●
●
●
●
●
●
●
●
3rd party sdks
agile/lean
backend services
competition
continuous
integration
deployment
distribution
device state
device fragmentation
●
●
●
●
●
●
●
●
●
●
platform fragmentation
in the wild
KPIs
location
mobile networks
review feedback
security
tools
user interface
web vs native vs hybrid

8. app quality attributes
●
●
●
●
●
●
●
●
●
●
content
elegance
interoperability
performance
pricing
privacy
satisfaction
security
stability
usability

9. keys to quality apps
1.
2.
3.
4.
5.
6.
7.
care about fixing your bugs
instrument your app
quality: perception, stars, feedback
prepare for changes like iPhone5 and iOS7
dogfood, beta, testing in the wild
watch competition
hybrid vs native question

10. fix known issues

11. instrument your app

12. popular. basic web and app performance too.

13. most popular with mobile-first and mobileonly

14. perception is important
brand
download

15. perception is important
trend: ask user for feedback.
if good: point to appstore
if bad: custom feedback
system

16. perception is important

17. changes
be ready
iphone users
especially fickle

18. competition
one click away

19. top tests that find bugs
Landscape/portrait orientation
Slow network, turning network on/off, switching wifi/LTE
Lots of data/content
First launch of app with no network or location or social
auth
App handling incoming call or other system interruption like
battery warning during data entry, device lock, etc

20. top tests that find bugs
Background processing
Lots of usage (flow issues and mem usage)
Multi-touch gestures/zooming
Correctness of branding and language used within the app
vs desktop and web.
Custom login—basic form validation
Other apps in the background

21. test automation
wait for design stabilization
wait for traction
framework? no great ones.
suggest opensource and
real-device compatible...
if you automate, use cloud
devices
monkey’s for android frag

22. dogfood, beta, crowd

23. hybrid vs native answer

24. freq updates != quality

25. future of app quality
real-time automated and manual test results
tools for engaging users/feedback
quantified
quality tied directly to engagement, $,
downloads
automatic suggestions (turing test for test lead)

26. future of app quality

27. app quality workshop
all day workshop for SASQAG tomorrow btw

28. contact
jarbon@gmail.com
@jarbon
always love to chat about app quality