We are increasingly moving towards mobile devices to fulfil our day-to-day computing needs. More smartphones are sold than PCs but many people are unclear on what changes to test strategies are needed when working with mobile. This presentation gives an overview of mobile testing, explains some of the common mistakes that are made when starting to test mobile, and gives the audience a taster of what to study next.

1. © Stephen Janaway 2015
MOBILE TESTING,
THAT’S JUST A
SMALLER
SCREEN, RIGHT?
STEPHEN JANAWAY

2. © Stephen Janaway 2015
WHO AM I?
STEPHEN JANAWAY
www.stephenjanaway.co.uk
@stephenjanaway
www.testinginthepub.com
@testinginthepub

3. © Stephen Janaway 2015
TODAY
• Is This Even Important Anyway?
• Isn’t Mobile Testing Just Like Any Other Sort
Of Testing?
• What I Think Is Different.
• What Makes a Great (Mobile) Tester.

4. © Stephen Janaway 2015
WHY
IS THIS
IMPORTANT?

5. © Stephen Janaway 2015
© Stephen Janaway 2015
MOBILE DEVICES ARE
MORE THAN JUST DEVICES
• Alarm Clocks
• Maps
• Link to loved ones
• Hand held shops
• Tickets
• Music Players
• Books
• Games Players
• Cameras
• ……

6. © Stephen Janaway 2015
MOBILE
PROJECTS OFTEN
DON’T HAVE
TESTERS…

7. © Stephen Janaway 2015
BUT SOON MOST
OF YOU
COULD BE
MOBILE TESTERS
ANYWAY

8. © Stephen Janaway 2015
© Stephen Janaway 2014

9. © Stephen Janaway 2015
SUBSCRIPTIONS VS. PEOPL

10. © Stephen Janaway 2015
A LOT IS
THE SAME
USE YOUR EXISTING SKILLS

11. © Stephen Janaway 2015
SIMILARITIES
• Functional testing.
• Non-functional
testing.
• Asking questions.
• Discovering
information.
• Communication.
• Understanding and
exploring the
software.

12. © Stephen Janaway 2015
DIFFERENCES
• Projects are quick.
• You can be closer to
design.
• Usability matters.
• You’re not sat at a
desk all day.
• Hardware influences
software more.
Testers as customer
advocates.

13. © Stephen Janaway 2015
PROJECTS
ARE QUICK
• CONSIDER EXPLORATORY APPROACHES
• CONSIDER CONTEXT
• DON’T EXPECT REQUIREMENTS (TO HAVE
ALL THE ANSWERS)

14. © Stephen Janaway 2015
GET
CLOSER TO
DESIGN
• REVIEW WIREFRAMES
• PAIR WITH UI DESIGNERS
• UNDERSTAND THE UI LANGUAGE
• USABILITY MATTERS

15. © Stephen Janaway 2015
USABILITY MATTERS
• Smaller screen.
• More difficult to
interact with and
typically used one
handed.
• How easy is it to use
the application? Do
user ‘just get it’?
• How is the look and
feel?
• Does the application
or device feel fast?

16. © Stephen Janaway 2015
USABILITY MATTERS

17. © Stephen Janaway 2015
GET OUT OF
THE OFFICE
• GO WHERE YOUR CUSTOMER ARE

18. © Stephen Janaway 2015
DON’T JUST TEST HERE…

19. © Stephen Janaway 2015
ON HERE, AND AT HERE…

20. © Stephen Janaway 2015
OUT HERE…

21. © Stephen Janaway 2015
IN HERE…

22. © Stephen Janaway 2015
USE THE
LATEST
DEVICES
• UNDERSTAND THE MARKET
• READ THE TECH BLOGS
• IT’S CONSUMER ELECTRONICS!

23. © Stephen Janaway 2015
© Stephen Janaway 2015
MOBILE DEVICES ARE
• Smart Phones
• Feature Phones
• Tablets
• E-readers
• Big screens, small screens, touch screens
and keypads.

24. © Stephen Janaway 2015
© Stephen Janaway 2015
AND NOW…
• Fitness devices.
• Smart watches.
• Wearable devices.
• Beacons.
• Household gadgets.

25. © Stephen Janaway 2015
MOBILE DEVICES ARE
COMPLICATED
While your application
runs, the phone is…..
• Talking to the
network.
• Working out where
you are.
• Maybe receiving
calls, texts, Facebook
and Twitter updates.

26. © Stephen Janaway 2015
CUSTOMER
ADVOCATES
• UNDERSTAND YOUR USERS - THERE ARE
LOTS OF THEM
• THEY ARE DIFFERENT
• THEY LIVE IN DIFFERENT PLACES
• EMOTIONAL ATTACHMENT

27. © Stephen Janaway 2015
?

28. © Stephen Janaway 2015
HOW DO YOU BECOME A
CUSTOMER ADVOCATE?
• User Persona’s.
• User Profiling.
• Industry Statistics.
• Apps Store
Reviews.
• Analytics.
• Use Your Users.

29. © Stephen Janaway 2015
BECOMING A
GREAT MOBILE
TESTER

30. © Stephen Janaway 2015
THE OBVIOUS STUFF
• Flexible.
• Great understanding
of technology.
• Employ exploratory
approaches.
• Coach the team on
how to deliver good
quality.
• Perhaps do less
actual testing?
• Give great
information in bug
reports.

31. © Stephen Janaway 2015
UNDERSTAND YOUR TECH
• OS updates.
• New devices.
• Using SDKs,
Android, Xcode.
• Build and deploy the
software.
• Battery life.
• OS specific
functionality.

32. © Stephen Janaway 2015
© Stephen Janaway 2014
DIFFERENT OS’S

33. © Stephen Janaway 2015
© Stephen Janaway 2014
DIFFERENT MOBILE
BROWSERS

34. © Stephen Janaway 2015
Source: https://developer.android.com/about/dashboards/index.html
ANDROID VERSIONS

35. © Stephen Janaway 2015
IOS VERSIONS
Source: https://mixpanel.com

36. © Stephen Janaway 2015
SCREEN SIZES

37. © Stephen Janaway 2015
USE REAL HARDWARE
• Can be dropped.
• Can get wet.
• Heats up!!
• Shaking.
• Tactile.
• Things are
removable.

38. © Stephen Janaway 2015
CHOOSE CLEVERLY
Open Device Labs
• Borrow devices,
normally for free.
• http://opendevicelab.
com
• Open Device
Labs = shared
community pools of
internet connected
devices for testing
purposes.

39. © Stephen Janaway 2015
USE
AUTOMATION
OR KNOW SOMEONE WHO DOES

40. © Stephen Janaway 2015
LEARN HOW
TO TEST WEB
SERVICES
DON’T JUST TEST THE CLIENT

41. © Stephen Janaway 2015
UNDERSTAND WHY APPS
GET REJECTED
More info needed
Bugs
UI Too Complicated
FraudulentName too similar
Place holder text
Inappropriate ratings
Beta/ demo

42. © Stephen Janaway 2015
WRITE GREAT BUG
REPORTS
Can I repeat the issue?
What steps are needed to reproduce the issue?
Is the issue repeatable on a fresh install of the application?
What data is available?
Which device and OS version was I using?
How was the device connected to a network (which
mobile operator and/ or Wifi network)?
Where was I when I saw the issue
What other applications were running?
Do I have screen shots or
a video of the issue?
Have I spoken with the developer?

43. © Stephen Janaway 2015
WORK MORE
CLOSELY WITH
THE USERS
HARNESS THE CROWD

44. © Stephen Janaway 2015
© Stephen Janaway 2014
HARNESS THE CROWD
• Usability testing.
• Internal user
groups.
• Beta test groups.

45. © Stephen Janaway 2015
ENGAGE USERS
THROUGH APPS STORES

46. © Stephen Janaway 2015
DON’T FORGET
ABOUT
SECURITY

47. © Stephen Janaway 2015
© Stephen Janaway 2015

48. © Stephen Janaway 2015
MONITOR APPS
IN LIVE
• USAGE PATTERNS
• CRASHES
• LOAD
• APP STORE RATINGS

49. © Stephen Janaway 2015
SO…

50. © Stephen Janaway 2015
SUMMARY - BUILD ON
WHAT YOU KNOW
• Customer advocate:
• Which platforms and devices will they use?
• How will they use the app or website?
• Mobile tech obsessive:
• Devices, OS versions, what’s being updated, what’s
new.
• Test using simulators, real devices, web
services and users.
• Understand the cost of automation.
• Use an exploratory approach to manual testing:
• Mobile device projects are typically short.

51. © Stephen Janaway 2015
EVERYONE IS
RESPONSIBLE
FOR QUALITY
IF WE DON’T FORGET THAT THEN IT’LL BE OK

52. © Stephen Janaway 2015
QUESTIONS
“Be proud, your work will soon
be in the hands of millions”
A NOKIA PRODUCT MANAGER
Stephen Janaway
www.stephenjanaway.co.uk
@stephenjanaway
www.testinginthepub.com