Contenu connexe Similaire à It's a Phone First! How to Test Your Five-star Mobile Apps (20) It's a Phone First! How to Test Your Five-star Mobile Apps1.
BW4
Concurrent Session
11/7/2012 10:15 AM
"It's a Phone First! How to Test Your
Five-star Mobile Apps"
Presented by:
Will Hurley
Quality and Security Services
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888‐268‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
2. Will Hurley
Quality and Security Services
A practice leader of quality management and security for Idea Integration Corporation, Will
Hurley has been at the crest of numerous software development waves during his twenty-four
year career. Will represented the US Air Force in the DoD’s first joint service system
architecture, led CMM assessments, and supported the development of the CMMi. After leaving
the Air Force, he consulted for industry-leading companies, helping them adopt and apply new
development methods. In his current position, Will consults with clients in North America on
achieving desired system and product lifecycle characteristics.
4. 28/09/2012
Since 1986, we’ve helped some of the world’s
largest and most respected brands thrive in the
digital space.
space
3
©2012 Idea. All rights reserved
The Presenter
Will Hurley
Practice Executive
Quality Mgmt. & Security
Idea
William.Hurley@idea.com
(781) 831-0389
©2012 Idea. All rights reserved
USAF Systems Engineer
TBMCS Core Systems Lead
Joint Service Architect GCCS
USAF Data Administrator
Consultant
Small Business Owner
Practice Leader
4
2
5. 28/09/2012
Acknowledgements
This discussion captures mobile testing insights developed at Idea’s
Global Solution Centers in Nashua, NH and Houston, TX.
Idea’s insights are supported by more than 10 y
years of mobile
g
pp
y
development and testing.
Insights are not SDLC or tool specific.
Data is updated on the quarter through a verity of sources.
ComScore®
Netbiscuits®
Harris Interactive®
OPA
Crittercism
5
©2012 Idea. All rights reserved
5
5 Star Mobile applications
6
©2012 Idea. All rights reserved
6
3
6. 28/09/2012
5 Star Mobile radar chart
©2012 Idea. All rights reserved
7
Ecosystem insights
Understanding the mobile ecosystem and trends increase test effectiveness
and product quality characteristics.
o
Capture more than 80% of smartphone users
with 3 network carriers.
ih
k
i
o
Capture more than 85% of smart phone
users with Android and iOS platforms.
o
Application use is growing faster than
browser use.
o
Android users spend more time on carrier
networks.
o
iOS users spend more time on wireless
networks.
©2012 Idea. All rights reserved
8
4
7. 28/09/2012
Ecosystem carrier
US Carrier Smart Phone Market Share
As of Aug 2012, 114 million Americans
own smart phones.
p
Verizon
31.60%
60 percent of all mobile devices acquired
in the U.S. are smartphones.
Verizon market share has grown fastest
driven by iOS.
Other 15%
AT&T
26.30%
Consider market share when signing
contracts for cloud services.
T‐Mobile
9.50% Sprint
9 50% Sprint
10.50%
Tracfone
7.10%
9
©2012 Idea. All rights reserved
Ecosystem oem
Samsung Galaxy S III leading 2012
sales may be passed by iPhone 5.
US Smartphone OEM
iPhone 4 took the top US device
sales spot in 2011.
Samsung,
25.6%
Other,
15.8%
iPhone 3GS took 2nd place.
LG, 18.4%
iPhone 4S took 3rd place.
BlackBerry Curve 8530 4th place.
RIM, 6.3%
Apple,
16.3%
16 3%
HTC Evo 4G 5th place.
HTC, 6.4%
Motorola,
11.2%
HTC is the most active OEM for
Android smartphones.
10
©2012 Idea. All rights reserved
10
5
8. 28/09/2012
Ecosystem platform
US OS Share
Other
0.5%
Android
52.2%
Symbian
0.8%
Microsoft
3.6%
RIM
9.5%
Apple
33.4%
66 percent of Americans ages 24-35
own a smartphone
smartphone.
Apple and Android market share are
growing.
Market share for all other platforms is
shrinking.
Android devices out number iPhones
by 3-to-2 in the U.S. market.
11
©2012 Idea. All rights reserved
Platform fragmentation
Fragmentation is a real but
manageable issue for all platforms.
A d id f
i
Android faces more i
issues since
it resides in an open ecosystem.
Android OS adoption is slower
than iOS.
Trades reporting >½ of all iPhone
users that can upgrade to iOS5 have.
iOS6 expected to drive largest
upg ade population date
upgrade popu at o to date.
Crash data are normalized as crashes as a percentage of each application launch.
©2012 Idea. All rights reserved
12
6
9. 28/09/2012
Ecosystem tablet
Tablet Share (Platform)
Other
12%
iPads account for more than 70% of
tablet network traffic.
Apple
46%
More than 80% of tablet traffic is wireless,
not cellular.
Tablet adoption is balanced between
males and females, trending older with
households earning $50K or more.
Android
42%
Kindle Fire
28%
An estimated 74.1 million consumers are
using tablets.
iPad 2 31%
©2012 Idea. All rights reserved
13
Customer Experience insights
Mobile users expect more.
o
Good design is not accidental. Get help if
you need i
d it.
o
o
Good design is sticky.
o
Analytics portray the customer voice.
Unstructured or unsolicited feedback has
more action value than a survey.
©2012 Idea. All rights reserved
14
7
10. 28/09/2012
Experience components
How easy is it for your customer to use
your application.
The application does
what your customer
wants it to do.
How does the
application make
li ti
k
your customer feel.
15
©2012 Idea. All rights reserved
Customer experience
Smartphone Usage
Music,
24.50%
Games,
31.80%
Social net
or Blog,
35.70%
©2012 Idea. All rights reserved
Text
Text
message,
74.60%
Used
browser,
48.60%
Used App,
48.60%
Four in ten mobile users are disappointed
with applications from their favorite
brands.
Three-quarters of users surveyed believe
a company's mobile app should be easier
to use than its Web site.
Users spend more time with mobile
applications than their mobile browser.
82% of iOS and Android Apps crash once
out of every 10 loads
Less than 5% of mobile apps are still in
use 3 months after downloading.
16
8
12. 28/09/2012
Experience themes
Based on conversations, we found several
common themes:
• All purchasing because it’s required by their
apartment complex
complex.
• All participants were referred to site via a
brochure.
• Participants pleased overall with the
simplicity of the online process
• Liked being able to select from presented
options in order to simplify and speed up the
process.
• Appreciated the ability to get additional
details where necessary.
• A majority didn’t use these features.
• All participants pay annually for their
policy.
©2012 Idea. All rights reserved
19
Experience maturity
o Functional requirements are only part of the customer
o
o
o
o
experience.
Key moments in customer interaction demand good
design.
If you don’t have experienced designers on your team
bring them in.
Understand accessible and emotional goals at every key
moment.
Don’t predict how people will respond to a new
experience,
experience test it
it.
o Allot the time to make incremental changes on all key moments.
©2012 Idea. All rights reserved
20
10
13. 28/09/2012
Functional insights
It’s a phone first.
o
Test while preforming common smart phone
tasks.
k
o
It’s cheaper, faster and easier to test in
emulated environments first.
o
Testing types are the same as Web based ntier applications
o
Manage functional and non-functional
insights across p j
g
projects.
o
o
Create a mobile test strategy document.
Reduce tool and process resistance
between groups.
©2012 Idea. All rights reserved
21
Manage scope
Carriers
Operating
Systems
OEM
Devices
Com &
Sensors
©2012 Idea. All rights reserved
22
11
14. 28/09/2012
Manage knowledge
©2012 Idea. All rights reserved
23
Manage modularity
•
•
•
•
•
•
•
•
•
Display and Graphics
Input Devices
Sensors
Permissions
Intents
Phone and messaging
Deploy and Upgrade
Analytics
A l i
Your application’s key moments
©2012 Idea. All rights reserved
24
12
16. 28/09/2012
Automated testing
Browser automation
Tools can be used for ‘closest‐match’,
browser testing on or off the device.
‘Closest‐match’ browser automation
possible on some devices ‐ Safari 3
for the iPhone driven on a PC.
User‐Agent header data to drive
compatibility testing logic.
Work closely with development team to
manage and control Web 2.0 features.
Third party libraries can speed
implementations.
Physical devices and Cloud services can be
used to test physical platforms when
defects are detected.
27
©2012 Idea. All rights reserved
Automated testing
Application automation
Tools can be used in emulated
environments and on the device.
Installation of some type of
automation controller is required on
the device.
Controllers consume resources.
Plug‐in and Cloud support is growing
and stable.
Blu
Ne eto
ar oth
Fie /
lld
Cloud services may not provide
object recognition.
Additional fixtures may be required for
sensor (GPS, Accelerometer) and
hardware (camera, battery) automation.
Applications may incorporate browser
engine.
©2012 Idea. All rights reserved
28
14
17. 28/09/2012
Functional details
Create a mobile test strategy document.
Tight control of manifest and OS support.
Identify development and testing tool stack.
Explain how you plan to manage fragmentation – OS, device, sensors, UI, marketplace.
Test teams should be working in emulated, physical and cloud based
environments.
Test teams should manage insights across releases.
HTML5 platform insights
Carrier and Device insights
OS / Platform insights
Security insights
Marketplace
©2012 Idea. All rights reserved
29
Security insights
Protect your brand.
o
Address security issues programmatically
and start now!
o
Static analysis is a highly effect approach to
address coding issues.
o
Dynamic analysis, Security Penetration
testing, is your last line of defense.
o
SD memory should be tested
©2012 Idea. All rights reserved
30
15
18. 28/09/2012
Security testing
Static Application Security Testing (SAST)
Dynamic Application Security Testing
Static analysis tools can address unsafe
Static analysis tools can address unsafe
practices earlier in the SDLC.
Clang; AppValidator; AnalytiX
SAST should be integrated with build and
reporting practices.
Evaluate application permissions.
Mobile devices are susceptible to
Mobile devices are susceptible to
common browser exploits and more.
Emulators or devices can be used to
ensure applications don’t bleed sensitive
data on to the device or to the network.
Keep up to date with known exploits and
add test cases to qualify the impact.
Don’t build on or deliver on compromised
platforms.
platforms
©2012 Idea. All rights reserved
31
Performance insights
Protect your brand.
o
Static analysis is a effect approach to
address performance issues.
p
o
Dynamic analysis, Security Penetration
testing is your last line of defense before
delivery.
o
SD memory should be tested.
©2012 Idea. All rights reserved
32
16
19. 28/09/2012
Performance testing
Static Application Performance Testing (SAPT)
Performance Testing
Static Analysis tools find issues earlier in
Static Analysis tools find issues earlier in
the SDLC and improve coding practices.
Load stress and longevity testing are
Load, stress and longevity testing are
critical to the customer experience.
Tools address code that slow application
performance.
Load and stress testing are focused on
middle and backend service layers.
In a 53K line project we found 328
instances of code that would
contribute to performance issues.
SAPT should be integrated with build and
reporting.
Longevity testing should look at both
backend services and the mobile device.
Performance tools should support
bandwidth throttling and common RESTful
protocols.
©2012 Idea. All rights reserved
33
Final thoughts
o
Cloud based testing services are valuable
but…
o
Will not achieve expected ROI without
proper investigation.
investigation
o
o
Lead to Scrumfall application lifecycles.
Can slow delivery times.
o
Functional, performance, security and test
automation should be considered for each
mobile project.
o
Simplify the testing space through active
p y
g p
g
management.
o
o
Diagram test environments.
Support agile approaches for all team
members.
©2012 Idea. All rights reserved
34
17