Over last couple of years, mobile devices have shown a phenomenal growth, at the same time PC industry is on a declining path. Due to this, we are experiencing a paradigm shift on how applications are built, tested, and used by the end users, and has a potential to create a disruption in the tradition way of software development and QA. Though it started with consumers, enterprises are also embracing mobility more and more, especially after the stupendous success of iPad. At the same time enterprises are also facing challenges in the area of provisioning, data management, device management, and security. Mobile devices are also used differently compared to traditional computing platforms. Due to this mobility devices and platforms throw up a new kind of challenges to testing fraternity. Calsoft Labs, with its unique competence and experience of working with leading Software and Hardeware companies, has been in the forefront of mobility. Calsoft Labs’ Mobility & Testing practice have been working together for some time to build methodologies, processes, and frameworks to address the challenges arising because of the above mentioned challenges shift and to create a new paradigm in application and product testing.

1. Future of Testing – Impact of Mobility Devices
Somenath Nag
24th March, 2012
Copyright © 2011 Calsoft Labs

2. • How is Mobility Changing the World
• Market Opportunity
• Challenges in Testing Mobility Applications
• Addressing the Challenges

3. Can we think of a day in our life without cellphone?
COMMUNICATION
UTILITIES/ TOOLS
PRODUCTIVITY
ENTERTAINMENT

4. Same is true for enterprises also
Execs
Line of Business
Managers
Task and Business Users
Consumers and Ecosystem
Mobile E-Commerce Mobile Marketing Self-Service

5. Enterprises are adopting mobility faster than ever
Numbers and types Mobility is driving the Increasing demand
of devices are “consumerization” of for enterprise
changing dramatically IT applications
• 1 billion • 46% of large • 20% of mobile
smartphones and enterprises workers are getting
1.2 billion mobile supporting business apps from
workers by 2013 personally-owned app stores today
• Large enterprises devices2 • 50% of
expect to triple • Billions of organizations plan
their smartphone downloads from to deploy mobile
user base by 2015 Apple App Store apps by 2012
Source: Gartner, IDC

6. When every industry is worried about
recession, mobility is growing at a supersonic speed
585,000+ Apps
25,000,000,000+ Downloads
400,000+ Apps
10,000,000,000+ Downloads

7. The unique challenges in testing mobile applications
• Diversity in Device Management
• Application Usage
• UX Design
• Network consideration
• Application & Device security
• Rapid Application Testing

8. Challenge 1 - Diversity in Device Management
• Different application platforms
– Each platform has its own standards, application interface, Programming
languages
– Needs to test applications on each platform for each features
– Different certification process for different platforms
– Increases time, cost, and effort
• Device factors
– No standardization of device & form factors
– New innovations creating more complexities
– Following is a list of the factors that can affect your application behavior:
• Screen size
• Android OS versions
• CPU

9. Challenge 2 - Paradigm Shift in Application Usage
PC Application Mobile Application
• Indoor use • Anytime, Anywhere
• Application dependent • Context Sensitive
• Data entry through key-board • Complex data entry process (key-
pad, touch, voice, etc.)
• Low or zero interruptions • High interruptions
• Low environmental input • High environmental input
(Camera, voice, etc.)
• Information Processing through • Information processing through
normal funnel an inverted funnel

10. Challenge 3 – Evolving Usage Patterns and innovations
are creating new testing challenges
• No established mobile software usability culture –
different for different platforms
• Direct usage of usability principles used in PC based
applications
• Impact of smaller form factors on usability
• Handling interruptions
• Interaction with other native and third party
applications (camera, voice, music players)
• Complexity created by gestures, multi-touch and
screen rotation

11. Challenge 4 - Network Considerations
• Unlike the network landscape of the PC
environment, the network landscape of a mobile
device may have gateways
• Applications should be tested for their compatibility
with WAP-enabled as well as HTTP-enabled devices
• Unpredictable network latency leading to
inconsistent data transfer speeds.
• Gateways in a wireless network may act as data
optimizers that may result in decreased
performance for heavy traffic

12. Challenge 5 - Application and Device Security Threat
Model
Courtesy: OWASP Project

13. Challenge 5 - Application and Device Security Risks that Need to be Tested
Top 10 Security Risks
• Insecure Data Storage
• Improper Session Handling
• Weak Server Side Controls
• Security Decisions Via Untrusted Inputs
• Insufficient Transport Layer Protection
• Side Channel Data Leakage
• Client Side Injection
• Broken Cryptography
• Poor Authorization and Authentication
• Sensitive Information Disclosure
13
Courtesy: OWASP Project

14. Challenge 6 – Rapid Application Testing
• In order to deliver the benefits of faster time to
market, RAD environments are used for mobile
application development.
• Since the time taken for development is reduced by
the introduction of RAD tools, builds will be
available for testing much earlier
• RAD methodology imposes an indirect pressure on
testing teams to reduce the testing cycle time
without compromising quality and coverage
• This pressure becomes more when applications are
targeted for multiple platforms simultaneously

15. A Mobility Testing Strategy Framework for addressing these challenges
Network Device Usage Performance Application
Landscape Landscape Scenarios Benchmark Category
Automation Consideration
Security
Consideration
(WOASP)
Test Strategy P Data Centric Apps
Device Capability
O based Apps
Test Execution
• Use a Weighted Device Platform Matrix method to identify the most critical
hardware/ platform combination to test
• Check the end-to-end functional flow in all possible platforms at least once.
• Conduct performance testing, GUI testing, and compatibility testing using actual
devices.
• Measure performance only in realistic conditions of wireless traffic and user load.
• Select the right automation test tool

16. Suggested Guideline for Tool Selection
• Support for ALL object identification methods
• Secure solution
• Ability to test devices taken out of a retail store - No
Jailbreak/Rooting
• Support for all device functionalities such as
gestures, security alerts, wake and reboot
• Plugs-in to existing test environment
• Same test should run on different devices

17. Q&A

18. About Calsoft Labs & Alten
Calsoft Labs ALTEN SA
■ Wholly owned subsidiary of ALTEN ■ Established in 1988
■ Outsourced Product Design arm of the ALTEN ■ European Leader in Engineering and
Group Technology Consulting services
■ Pioneer in Product Engineering Services ■ 14800 employees worldwide, >90%
engineers and consultants
■ Over 300 satisfied customers
■ Located in 16 countries
■ Over 1000 products developed and tested
■ Revenue of € 1 Billion+ for FY 2011
Service Offerings Industry Focus People Locations
• Embedded Product • Telecom • 15000+ people • Located in 16 countries
Engineering Service • Healthcare • 65% of engineering team • 10 Delivery centers
• Software Product • Education with 5 years+ experience • Design & Development
Engineering Service • Enterprise Software • 27% Master Degree Centers in
• Independent Verification • Automotive holders Bangalore, Mysore &
& Validation Service • Media & Entertainment • ALTEN is one of Europe's Chennai
• Design Engineering • Semiconductor most dynamic • US Delivery Centre in
Service • Aeronautics / space employers, hiring almost Boston, MA
• Financial 2,000 new people each
year.
Our Partners

19. Calsoft Labs’ Testing Practice
Partner
■ 200+ dedicated test professionals
■ Track record
■ QA solutions throughout the SDLC based on TMMi framework
■ Reusable repository of test cases, scenarios, and validation libraries
Service Offerings Industry Focus Thought Leadership Mobility Testing
• Test Strategy Consulting • Telecom • OWASP Compliance • Field testing and
• Embedded Product • Healthcare • Accelerator for OWASP Interoperability
Testing • Education Compliance • Carrier validation test
• Software Product Testing • Enterprise Software • Home-made Open Source • Protocol and
• Test Automation • Automotive Automation Framework Conformance test
• ‘On-Demand’ Testing • Media & Entertainment • Device Management
• Compliance testing • Semiconductor Testing
• Security Testing • Aeronautics / space • Mobile App testing
• Financial • Usability Testing