Build Your Mobile Testing Expertise

MO
PM Tutorial
9/30/2013 1:00:00 PM

"Build Your Mobile Testing
Expertise"
Presented by:
Karen Johnson
Software Test Management, Inc.

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

Karen N. Johnson
Software Test Management, Inc.
Karen N. Johnson is a software test consultant, working on-site and remotely to help her clients
build testing teams and strengthen their testing practice. Karen is considered an expert in
software testing and has been part of the context-driven software testing community for a
number of years. She is a frequent speaker at testing conferences; a contributing author to the
book, Beautiful Testing; and co-founder of WREST, the Workshop on Regulated Software
Testing. Karen has published numerous articles and blogs about her experiences with software
testing.

8/20/2013

Build Your Mobile Knowledge
section 1

Build Your Mobile Testing Expertise v7

© Karen N. Johnson, 2013

1

agenda
•
•
•
•

device selection
UX
device settings
competitive analysis

Build Your Mobile Testing
Expertise v7


2

1

8/20/2013

One thumb design
Fat Thumb: A One-Handed
Alternative To Pinch-ToZoom
http://www.fastcodesign.c
om/1671741/fat-thumb-aone-handed-alternative-topinch-to-zoom#1
Testing One Thumb, One
Eyeball Mobile Use
http://www.lukew.com/ff/
entry.asp?1664



3


4

Touch Gesture Icons
http://www.mobiletuxedo.com/t
ouch-gesture-icons/
Input Guidelines
http://uxdesign.smashingmagazin
e.com/2011/10/06/not-yourparents-mobile-phone-ux-designguidelines-smartphones/
Touch Gesture Reference Cards
http://www.lukew.com/ff/entry.a
sp?1370


2

8/20/2013

Twitter

Content
Karen McGrane @karenmcgrane
Deane Barker ‫ ‏‬gadgetopia
@
cleveg ‫@‏‬cleveg
SaraWachterBoettcher ‫@‏‬sara_ann_marie
UX: Theresa Neil ‫@‏‬theresaneil (watch her website
and slideshare)
RWD
Ethan Marcotte ‫@‏‬beep
Responsive Design ‫@‏‬RWD
Jordan Moore ‫@‏‬jordanmoore

Mobile
Brad Frost ‫@‏‬brad_frost
Brian Fling ‫@‏‬fling
Jason Grigsby, ☁4 ‫@‏‬grigs
Jonathan Kohl ‫@‏‬jonathan_kohl
steven hoober ‫@‏‬shoobe01
Josh Clark ‫@‏‬globalmoxie



5

Content Everywhere
by Sara Wachter-Boettcher
Books
Jump Start Responsive Web Design
by Craig Sharkie & Andrew Fisher
Mobile First
by Luke Wrobleski
Mobile Developer's Guide to the Galaxy
see the Enough Software website



6

3

8/20/2013

device selection
part 1



1

Device Selection:
How will you decide which devices?
Possible Factors:
 budget
 procurement
 time
 usage statistics
 manufacturers & models
 operating systems
 network carriers
 m-commerce
 mobile browsers

Expertise v7







cookies
installation
responsive web design
input methods
location awareness & map
providers
 proprietary network
 barcodes and scanners
 flash


2

1

8/20/2013

Device Matrix
source: Mobile Design and Development, by Brian Fling
*** This is a model for making a decision. Does this model help you?

Class

Markup

CSS

Javascript

Class A
Class B
Class C
Class D
Class F



3

Budget, Procurement & Time:
Budget:
Can you afford the devices you want to test?
Procurement:
Can you find the devices?
Time:
If you resolve budget and purchase, would your team
have the time to test all of the devices?
•
•

Teams often solve the budget issue with using their own devices.
Teams often solve finding the devices through eBay or Craigslist.



4

2

8/20/2013

Mobile Statistics:
Do market statistics influence which devices to test?

•
•
•

Country: Usage statistics vary
greatly by country.
Source: Which source do you
trust?
Analytics: Are you capturing
device usage for your own
application?



5

Mobile Manufacturers: & Models:
Does manufacturers or specific models influence which
devices to test?
•
•
•

Manufacturers vary by country.
Devices are updated frequently.
How often do you update your
device coverage?



6

3

8/20/2013

Operating Systems:
How do you decide which OS's to cover ?
•

How do you address forced OS
updates?

•

How do you maintain older versions?



7

Network Carriers and Settings:
• How do you acquire
network simulation for
'other' countries?
• What about different
keyboards and languages?
• What about SMS text
testing?



8

4

8/20/2013

Does testing from the m-commerce perspective
impact device selection?



9

Mobile Browsers:
If you're testing on mobile browsers, how does that
influence device selection?



10

5

8/20/2013

Cookies:
Do cookies influence device selection?
• Does your app or site offer
cookies?
What about:
• Auto-login
• Third party login access (i.e.
Use Facebook account to
login)



11

Installation
Application install & updates



12

6

8/20/2013

Location Aware:
Does location aware influence device selection?
• How does map provider
selection influence device
selection?



13

Input Selector & Navigation:
Does the input method impact device selection?
•
•
•
•
•
•
•

Rotate
Touch
Click wheel
Rollerball
Pinch & zoom
Typing/keyboard inputs
Gestures

Build Your Mobile Testing Knowledge v7


14

7

8/20/2013

Mobile Proprietary Network Access:
Are you required to test particular devices?

•
•

Does your mobile testing need to
pass an audit or regulations?
Are there VPN considerations?



15

Responsive Web Design:
Do you need to check content and layout on different
devices?
•

People often use both at the same
time.

For more on RWD, see:
• http://blog.creativemagma.com/feat
ured/2209/

Build Your Mobile Testing Knowledge v7


16

8

8/20/2013

exercise: build a device matrix (or mind map)
•
•
•

With your team, determine what factors impact your decision.
Update the checklist of factors to reflect your decisions.
Build your own device matrix or mindmap to identify what devices to test.



17

Device Selection:
How will you decide which devices?
Possible Factors:
 budget
 procurement
 time
 usage statistics
 manufacturers & models
 operating systems
 network carriers
 m-commerce
 mobile browsers








cookies
installation
responsive web design
input methods
location awareness & map
providers
 proprietary network
 barcodes and scanners
 flash


18

9

8/20/2013

Device Matrix
source: Mobile Design and Development, by Brian Fling
*** This is a model for making a decision. Does this model help you?

Class

Markup

CSS

Javascript

Class A
Class B
Class C
Class D
Class F



19

10

8/20/2013

Mobile User Interface, Design &
User Experience
section 3



1

UX Testing:
How will you decide what to test?
Possible Factors:
 menu styles
 navigation
 input selection
 UI controls
 custom controls
 links
 lists
 colors & symbols
 currency

Expertise v7











images
video
completing forms
login
search, filter & sort
app version
sitemap
RWD
accessibility


2

1

8/20/2013

Menu Styles
Springboard


List


3

Menu Styles
Tab


Gallery


4

2

8/20/2013

Menu Styles
Carousel


Metaphor


5

Navigating
Giving the user a unique way to
move forward & back.



6

3

8/20/2013

Navigating
• Paging
• Forward & backward

• Cookie crumb trail
(websites)



7

Navigating
• Rotate
• Pinch/zoom
• Accelerometer
• Vertical/horizontal
• Paging forward & backward



8

4

8/20/2013

Input Selector
•
•
•
•
•
•

Rotate
Touchscreen
Click wheel
Rollerball
Pinch & zoom
Typing/keyboard inputs



9

UI controls
• Entry fields
• Lookup buttons
• Radio buttons
• Drop downs



10

5

8/20/2013

Custom Controls
• The slider bar on AA’s
mobile app is an unusual
custom UI control.
• Does your app have any
custom controls?



11

Links
• Do links work?
• Does your site have links to
other sites?
• What about the sitemap
links?



12

6

8/20/2013

Lists
• Alignment
• Consistency

• Disruptions from ads or 3rd
party feeds



13

Color-alternating lists
•

Grouped
rows

•

Zebra
stripping lists

•

Text
Alignment

•

User
Accessibility



14

7

8/20/2013

Colors & Symbols
•
•
•
•

Symbols
Spacing
Rotating
Columns



15

Currency
User interface considerations
with multiple currencies: is the
correct symbol for the
currency shown?



16

8

8/20/2013

Images
• Where’s the image?
• You might use your desktop
to see if the issue occurs
outside of the mobile
environment.



17

Images
•
•
•
•
•
•
•
•
•

Size of files
Cache
Refresh rate
Full screen
Wi-Fi drops connection
Rotate
Pinch/zoom
Accelerometer
Vertical or horizontal



18

9

8/20/2013

Video & Streaming
• Flash?
• Videos

• Connection speed
• Availability



19


20

Completing Forms
At some point, a user
needs to enter
information. Minimizing
input – such as offering
smart fill fields on location
awareness helps to cut
down on the amount of
information a user has to
enter.


10

8/20/2013

Completing Forms
• Sign-in and search are
two of the most
common forms on
mobile devices.



21

Login



22

11

8/20/2013

Passwords
• Case sensitive
• Changing the case

• Seeing the text entry
• Staying logged in (cookies)



23

Search
• Wildcards
• Text entry
• Case sensitivity
• Alpha and numeric entry
• Dropdowns
• Location based searching


24

12

8/20/2013

Search & Filter
• Location-based
filtering
• UX considerations
on filters, applying
& clearing
• UX considerations
– scrolling through
many search
results


25

Search & Sort
When results are pages
long on a mobile device,
offer the user a means to
sort or refine the result
set.



26

13

8/20/2013

Search
• Are there fields that are prefilled based on date, location or
time?
• OpenTable is an example of time and location.
• Are there fields that are not offered on mobile but are offered
on the desktop version?
• Is there a search button? Is there another method to start or
run the search?
• Is there a way to refine the results?



27

Search: filtering results



28

14

8/20/2013

A short lesson on
responsive design
http://blog.creativemagma.com/featur
ed/2209/



29


30

Phone vs. tablet
People often use both at the
same time.
Testing by planning user
scenarios is a good way to
uncover and identify
situations to test.


15

8/20/2013

App Version
• A version page is an
opportunity to once
again show the app icon
and/or company logo.
• What about other
maintenance pages?



31

Sitemaps
“They visually represent the relationship of
content to other content and provide a map for
how the user will travel through the
informational space.” Brian Fling, author Mobile Design and Development



32

16

8/20/2013

exercise: build a list of ideas for testing based on
UX factors
•
•
•

With your team, determine what UX factor influence testing
Update the checklist of factors to reflect your decisions.
Build your own checklist or mindmap to identify what devices to test.



33

UX Testing:
How will you decide what to test?
Possible Factors:
 menu styles
 navigation
 input selection
 UI controls
 custom controls
 links
 lists
 colors & symbols
 currency

Expertise v7











images
video
completing forms
login
search, filter & sort
app version
sitemap
RWD
accessibility


34

17

8/20/2013

Mobile App & Device Settings
section 4



1

app, device & browser settings
•
•
•
•
•
•
•
•
•
•
•
•
•

Overall app permissions
Specific app permissions
App permission-dependencies
Device settings
Browser settings
Cookies
Wireless & network settings
Data Roaming
SD card settings
Swype
Display settings
Battery Settings/Status
Task Killers



2

1

8/20/2013

Overall App Permissions
Know the app permissions
your mobile app requires.
Consider what the user may or
may not have enabled on the
device for all applications.



3

Specific App permissions
What permissions does
your application require?



4

2

8/20/2013

App permission-dependencies
If your app interfaces with 3rd
party data such as location
finding, what happens when
that access is off, down or
unavailable?



5

Device settings
• Consider the environment of
the device: what has been
enabled or not.
• Consider the ability or
restrictions to downloading
and installing applications.
• Consider upgrade scenarios
and whether permissions have
been changed or reset.
• Install testing has returned.


6

3

8/20/2013

Browser settings



7

Language settings
• Finding what languages are
available on the device

• Choosing a different
language
• Accessing and using a site or
an app that’s multilingual
• Using different keyboards
on devices



8

4

8/20/2013

Cookies
• Does your site use cookies?
• Check the login process
with cookies on/off.



9

Wireless & Network Settings
Have you ever tested your
website when connectivity is
down or disrupted? This
scenario is more likely with
mobile devices.



10

5

8/20/2013

Network Mode
CDMA is used in the US (also in China
and India) but everywhere else uses
GSM or other modes of connection.
It can be challenging to test like a
user when you live in a different
country and some options are not
available on devices where you are
located.



11

Network search
Users can switch from
and to an application or
browser session while
other activities are
taking place (including
incoming phone calls)
does your app handle
that disruption?



12

6

8/20/2013

Connecting to a network
A user could switch to
your app or website when
they’re not connected.



13

Connection errors
If your app uses the SD
card for storage – what if
the card is already full or
another app writes to the
card?



14

7

8/20/2013

Data roaming
Would you enable your
testing team to leave the
office and experience
mobile as they travel
around the office?



15

SD card settings
• Does your app store
data on a card?
• What if the card is
removed?
• What if a different card
is inserted?



16

8

8/20/2013

SIM cards
• A SIM card or Subscriber
Identity Module is a chip.
• SIM holds personal identity
information, cell number,
phone book, text messages
and other data.
• Again, what happens if the
card is full? Changed?
Removed?


17

Swype settings
• Does your app work
with swyping or voice
input?



18

9

8/20/2013

Display settings
Consider whether a user
has made device settings
that can impact how your
application appears.



19

Battery conditions
• Have you tested your
app under low battery
conditions?
• Is your application
responsible for battery
drainage?
-

See Julian Harty’s work
See Android Forum



20

10

8/20/2013

Battery conditions: power save
Is this a variable you need to
consider in testing?



21

Task Killers
•
•
•

What background tasks are
running?
What data is being synced?
Does a Task Killer help or not?

Android
http://www.pcworld.com/article/24
4067/taskkiller_apps_will_they_help
_or_hurt_your_battery_life.html
IOS
http://www.youtube.com/watch?v=
nEJA1yUgR80



22

11

8/20/2013

Exercise:
Device Settings

• What device settings make an
impact on the app you’re testing?

1. Identify what settings
need to be factored into
testing or configuring a
device.

• What app permissions does your app
require?

2. Build a mind map or
outline structure to group
these settings together in
a logical way.

• What settings make an impact on
mobile browser testing?

3. Should the settings be
grouped according to
device manufacturers?
Such as grouping Nokia
settings vs. Samsung
settings?


23

12

8/20/2013

The Competitive Mobile
Marketplace
section 5



1

What do mobile apps compete on?
According to uTest’s white paper from the product
Applause, there are intuitive metrics such as:
• content
• elegance
• pricing
• usability
• and satisfaction



2

1

8/20/2013

How steep is the competition?
•
•
•
•

Apple Store: 700,00+ apps
Google Play: 700,000+ apps
Blackberry App World: 100,000+ apps
By 2015, the International Data Corporation (IDC) predicts
182.7 billion mobile apps will be downloaded
• In 2010, 10.7 billion apps were downloaded
- source: uTest Applause white paper



3

MARKET RANKING



4

2

8/20/2013

Market Rankings
Applyzer is a product that watches
market rankings. Notice how they
segregate data by country and mobile
app category.



5


6

Market Rankings
Distimo is another market watching
application.


3

8/20/2013

Market Rankings



7


8

Market Rankings
RankedNow is another product
tracking the competitive market.


4

8/20/2013

MOBILE COMPETITORS



9

Competitive Analysis – Vertical Competition
mobile app to app
AccuWeather


The Weather Channel


10

5

8/20/2013

Competitive Analysis – Vertical Competition
tablet app to app



11

Competitive Analysis – Mobile Competition
the app Shazam vs. other ways to spend time

Notice how competitors might not
be offering the same product but
compete equally well for a user’s
time, purchase and use.



12

6

8/20/2013

Mobile competition:
Shazam’s “competitors”

See how this “competitor” isn’t offering
the same product but is vying for the
user’s time.

This competitor is a more typical
“competitor.”


13

Mobile competition:
a sample of differences by store

Competitors vary from store to store.



14

7

8/20/2013

MARKET COMMENTS



15

Charmin – Sit or Squat

This Charmin app was liked until the app was tied to Facebook and requires a FB
Account in order to login or comment. Users have scathing comments about the change.



16

8

8/20/2013

Tide Stain Brain

The Tide app made a change when the company removed user comments on how to
Remove stains and in turn infuriated the user community.



17

Chlorex myStain

The app gains respect and ratings by now being solely focused on pushing their product.



18

9

8/20/2013

Out of Milk

Note the number of users for the free version. And also note the overall ratings by users.



19

Spotify

Note how users comment on updates that cause issues or comment on missing
functionality.


20

10

8/20/2013

Competitive Analysis
Exercise:
1.

2.
3.
4.

5.

Select two mobile applications that
would seem to be competitors such as
AccuWeather and the Weather
Channel.
Review each app for the “intuitive
metrics” listed as factors.
Make notations on each of the factors
for both apps.
For satisfaction, you can use either your
satisfaction with the app, app store
comments or a combination of both.
If you were to apply a scoring to each
app, which app would you choose over
the other app?



21



22

11

Build Your Mobile Testing Expertise

Recommandé

Recommandé

Contenu connexe

Plus de TechWell

Plus de TechWell (20)

Dernier

Dernier (20)

Build Your Mobile Testing Expertise