For more info visit: www.zentestlabs.com
www.blog.zentestlabs.com
--Mobile Application Testing
--Mobile Configuration Optimization
--Mobile Test Automation
--Mobile Performance Testing
2. ABOUT ZEN TEST LABS
Domains
Technology, Tools and Frameworks
๏ผDevelopment & Testing Expertise
๏ผProduct Focus
๏ผISO 27001 & ISO 9001 certified
processes
๏ผDomain accelerators
๏ผROI driven engagement models
๏ผThought leadership- STAR
conferences, etc.
Geographic Spread
๏ผUSA operations based out of
Chicago and Maine
๏ผOffshore testing lab in India
๏ผAbility to deploy onsite resources
for key phases of projects
๏ผAbility to dip into resource pool to
fill onsite gaps
๏ผCustomer base that spans US, EU,
APAC and India
๏ผPartner network that spans US, EU,
MEA and APAC
2
๏ผMobile
๏ผGaming
๏ผCloud
๏ผBanking & Financial Services
๏ผBanking & other ISVs
๏ผeLearning
3. TYPICALCHALLENGES FACEDAND THEBUSINESS IMPACT
3
Common Challenges
โขOS & Device fragmentation
โข Rapid rollout schedule
โข Dealing with a wide range of connectivity modes
โข Security
โข Data Usage
โข Installation and launch
โข App Performance โ Memory and BW consumption,
behaviour under low bandwidth & memory
Business Impact
โข Having an app run on some devices & not others is a
major hindrance to widespread adoption.
โข Quick rollouts can become extremely expensive for
the companyโs reputation if the app doesnโt meet
the quality users expect.
โข Wi-Fi, 3G, 4G, all connectivity modes can
exponentially multiply the no. of tests to run and
become costly & time intensive.
โข Users expect personal data to remain personal &
expect apps to be 100% free of viruses, malware &
other threats.
โข Users abandon apps that drain battery faster in
favour of more economical ones.
โข 60% of mobile users abandon the app or website if it
doesnโt load within 3 seconds!
4. THE ZEN TEST LABSโ SOLUTION
4
MOBILE CONFIGURATION OPTIMIZATION
Zen Test Labsโ unique utility, provides the optimum no. of
configurations to test your app on in a few easy clicks.
MOBILE TEST AUTOMATION
Zen Test Labsโ automation methodology guarantees 60-70% reduction
in testing effort, complete coverage across devices & highly reusable
scripts.
MOBILE PERFORMANCE TESTING
Holistic approach to performance testing covering areas such as volume
testing, endurance testing, performance monitoring, soak testing and
testing under real time scenarios.
5. ZEN TEST LABSโ CONFIGURATIONOPTIMIZATIONUTILITY
5
WIDE CHOICE OF
DEVICES
โข Select based on
app. Type
(Native, Web,
Hybrid)
โข Select based on
device type (
Smartphone,
Tablet)
WIDE CHOICE OF
CONFIGURATIONS
โข Select OS and
Versions
(Android, iOS ,
Windows Phone
etc.)
โข Select device
manufacturer
(Samsung, HTC,
etc.)
OPTIMIZED DEVICE
COMBINATION
โข Prioritized &
optimized
combination of
devices to test
your app.
โข Predict
popularity of
devices in target
segments (geo.
specific)
6. ZEN TEST LABSโ CONFIGURATIONOPTIMIZATIONUTILITY
6
MARKET
RESEARCH
WIDE RANGE OF DEVICES
TO TEST
OPTIMIZED &
PRIORITIZED SET OF
DEVICES TO TEST
BI
7. ZENMOBILE: ZEN TEST LABS' MOBILEAUTOMATION FRAMEWORK
7
APPLICATION UNDER TEST
(PARALLEL EXECUTION ACROSS MULTIPLE
DEVICES WITH DIFFERENT OSโS)
AUTOMATION TOOL (COMMERCIAL: QTP
PLUG-IN OR OPEN SOURCE: MONKEY TALK)
ZENMOBILE AUTOMATION FRAMEWORK
TEST MANAGEMENT
TOOL
EXECUTE
BUILD
FUNCTIONS
CALL
FUNCTIONS
BUILD SCENARIOS
BUILD TEST CASES
BUILD TEST DATA
VIEW REPORTS
REPORT &
MANAGE
8. ZENMOBILE (FRAMEWORK) FEATURES
8
๏ผ Dashboard rich with features like displaying ongoing execution along with status, time required for
execution of individual business functions
๏ผ Parallel execution across multiple devices and configurations
๏ผ Development language agnostic
๏ผ Automation tool agnostic
๏ผ Runs on real mobile devices as well as emulators
๏ผ Provides performance analysis reports
๏ผ Test data reusability
๏ผ Ability to work in a continuous integration environment
๏ผ Ability to work with native, web based and hybrid apps
9. ZEN TEST LABSMOBILEPERFORMANCE TESTING
9
Test
performance
across variety
of devices and
environments
Load Testing
Stress Testing
Volume Testing
Soak Testing
Scalability Testing
Interruption Testing
Provide Metrics
Across
Resource Utilization
App. Behavior
Memory Leaks
Response Time
Test app.
performance
under varying
conditions for
bandwidth,
memory and
battery.
Gather Inputs
Establish Performance
Benchmark
Performance Goals
Critical Scenarios
Workload
10. ZEN TEST LABSโ MOBILETESTINGMETHODOLOGY
INPUT
GATHERING
โข Plan to release, current
status , geography
โข Define purpose of
testing.
โข Classify app according to
type(Native, Web,
Hybrid)
โข Establish the OS and
versions on which the
app will run
โข Establish the no. of
releases planned
โข Establish the budget for
the number of
configurations.
APPLICATION
DIAGNOSIS
โข Explore the application
โข Size the application
โข Identify best
configurations for the
app based on market
research
โข Identify the no. of rounds
required for testing
โข Design optimized test
cases
โข Identify readymade test
case repositories
โข Calculate total effort
required for testing
ROADMAP &
DELIVERABLES
โข Provide a roadmap for
testing the application
โข Provide a list of optimum
configurations required
for testing
โข Outline the total effort
required for testing
โข Provide value adds for
achieving better ROI
โข Provide an overview of
other non functional
tests critical to the
application
โข Deliverables
โข Identify if specialized
test are required
10
11. ZEN TEST LABS EXPERIENCE
๏ผ Test automation on the cloud for a US based leading e-gift card services provider
๏ผ Game testing for an iPhone based game development company
๏ผ Web testing of games and tools for a leading academic and reference books publishers
๏ผ End to end testing of a mobile data back up application for Mobibook (a mobile application
development company) across mobile platforms and social networks
๏ผ Manual testing for a web 2.0 based email marketing solution
๏ผ Web testing of games and tools for a global media conglomerate engaged in publishing, music,
magazines, TV and radio production and distribution, as well as Internet services
๏ผ End to end testing of multiple mobile and web based applications for a leading โuser-centricโ
software solutions company
๏ผ Web testing of games and tools for one of the largest US-based development house for learning
materials.
11
12. CASE STUDY
The Client
The client is a product based company, and they introduced an app as a new product line. The app enables the user to
read their favourite stories anywhere, anytime be it either books, magazines, blogs etc. The user can browse, search, and
download them on the mobile phone! They can re-download stories from My Bookshelf countless times and all data was
meant to be stored on Cloud. It also enables a cross platform multi-lingual chat .
Key Project Challenges
โข Constant crashing of the App due to instability
โข To be tested across various configurations considering wide range of devices with multiple OS versions
โข Limited bandwidth of resources which hamstrung the width of testing
โข Vendor issues with domestic and global ones for testing payment gateways
โข Frequent releases, since agile mode
โข Undocumented requirements and constant evolving of requirements
Our Solution
Zen Test Labs put together a team of 2 manual testers. In addition to this Zen Test Labs developed a mobile testing lab
consisting of the latest devices the client wanted the app to run on
โข Consulted at the business Requirements stage to structure and ensure coverage of the testing process
โข Performed End to End testing scenarios via user simulation testing
โข Performance testing of app on cloud with 10,000 users
โข Functional test automation and performance testing
โข Optimized device and OS configurations to ensure maximum output in minimum time
12
13. CASE STUDY
13
โข Acquired business objectives from the client
โข Expectations from the stakeholders
โข Challenges and methodologies from the developers
Inception
โข Milestones of the project
โข Aligned release dates along sprints
โข Wrote test cases which were used by developers to
ensure coverage and functionality
Plan
โข Executed on the basis of Agile
โข Performed user simulation testing and allowed โCarry
the device home'
โข Immediate reporting of bugs
โข Simultaneous fixing of bugs which resulted in the
defects being fixed before the next release went live,
<reduce the cycles>
Execution & Reporting
โข Performed Test automation and regression suits
followed by performance testing on the staging
environment which was a replica of the production
environment
โข Sign off
Testing on Production
Environment
Key Results
โข Logged 316 defects with
more than 25% falling
under the category which
were crashing the app
โข Automated a regression
suite of 250 test cases
โข Single click run facility to
run automated smoke test
cases within 30-45
minutes
โข Reduced 1.5 day of
manual execution to 2
hours along with the
facility of auto reporting
โข Simultaneous fixing of
bugs which resulted in the
defects being fixed before
the next release went live,
which in turn reduced the
cycles
โข Reopen ratio was
drastically reduced
Methodology