Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
 
W8
Session	
  
4/15/2015	
  1:00	
  PM	
  
	
  
	
  
	
  
"Mobile Test Automation with
Big Data Analytics"
	
  
Presente...
Tarun Bhatia
Microsoft
Tarun Bhatia is a technical program manager in charge of driving the best breed
of performance meas...
4/8/15	
  
1	
  
Tarun Bhatia
Mobile Test Automation Using Big Data Analytics
Introduction
quality	
  as·∙sur·∙ance:	
  
A...
4/8/15	
  
2	
  
Staged Rollout with Active Monitoring
•  Crash
Reports
•  User
Reviews
20%
User
Base
•  Crash
Reports
•  ...
4/8/15	
  
3	
  
Data Everywhere
Trends in Tech Salary Reaffirm
Source: http://marketing.dice.com/pdf/Dice_TechSalarySurve...
4/8/15	
  
4	
  
“
”
If you think you can,
or if you think you can’t,
you are correct. – Henry Ford
Question
Your confiden...
4/8/15	
  
5	
  
How it Starts!
Stage 1
•  Company needs mobile presence
•  They hire Mobile Devs andTesters (usually manu...
4/8/15	
  
6	
  
Creating a Device Lab
Creating a Device Lab (Using Big Data)
Total # of
Devices
Devices with
most # of
re...
4/8/15	
  
7	
  
Creating a Device Lab
30%
17%
13%
5%
4%
4%
3%
3%
3%
3%
2%
2% 2%
2% 2%
5%
Apple
LG MS770
Samsung Galaxy SI...
4/8/15	
  
8	
  
Prioritize
KPI
Customer
Usage Data
Finance
(Revenue
Stream)
Data
Marketing/
Social Data
User Usage Patter...
4/8/15	
  
9	
  
Tests
Real User, Marketing and Finance
Data
Stress
ServerVs. UI Data
New Features
Performance
System Unde...
4/8/15	
  
10	
  
Server Vs. UI Testing
Server
Client Test Framework
•  Verify	
  data	
  is	
  in-­‐sync	
  
during	
  te...
4/8/15	
  
11	
  
Effective Testing
Write
Once,Test
Anywhere
Active
Monitoring
Test Re-Use
Performance
Availability
Conclu...
Prochain SlideShare
Chargement dans…5
×

Mobile Test Automation with Big Data Analytics

273 vues

Publié le

Development and test organizations face major challenges when building robust automated tests around their mobile applications. With limited testing resources and increasingly more complex projects, stakeholders worry about the risk and quality of mobile products. So how do you plan a mobile test automation project to prioritize testing resources and efforts? Tarun Bhatia used big data analytics to understand where customers spend most of their time on their apps out in the wild. See how you can analyze massive amounts of mobile usage data to create an operational model of carriers, devices, networks, countries, and OS versions. Based on real-user data, they developed automation strategies to create better tests and focus on the right priorities. Learn how you can use big data analytics to apply mobile automation in areas of continuous integration, performance, benchmarking, compatibility, stress, and performance testing.

Publié dans : Logiciels
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Mobile Test Automation with Big Data Analytics

  1. 1.   W8 Session   4/15/2015  1:00  PM         "Mobile Test Automation with Big Data Analytics"   Presented by: Tarun Bhatia Microsoft                 Brought  to  you  by:         340  Corporate  Way,  Suite  300,  Orange  Park,  FL  32073   888-­‐268-­‐8770  ·∙  904-­‐278-­‐0524  ·∙  sqeinfo@sqe.com  ·∙  www.sqe.com
  2. 2. Tarun Bhatia Microsoft Tarun Bhatia is a technical program manager in charge of driving the best breed of performance measurements and analysis for Microsoft Online Office Division. Tarun leads innovative strategies—analytics, performance, benchmarking, and compatibility—and guides the team to create an effective, reliable, and robust monitoring architecture. With more than seven years of software development experience in quality and service assurance, Tarun shows that taking initiative and thinking outside the box can deliver big results—both personally and for the company.  
  3. 3. 4/8/15   1   Tarun Bhatia Mobile Test Automation Using Big Data Analytics Introduction quality  as·∙sur·∙ance:   A  program  for  the   systema<c  monitoring   and  evalua<on  of  the   various  aspects  of  a   project,  service,  or   facility  to  ensure  that   standards  of  quality  are   being  met       Source:  hCp://www.merriam-­‐webster.com/ dic<onary/quality%20assurance  
  4. 4. 4/8/15   2   Staged Rollout with Active Monitoring •  Crash Reports •  User Reviews 20% User Base •  Crash Reports •  User Reviews 50% User Base •  Crash Reports •  User Reviews 100% User Base Manage Analyze ExtractValue Value What is Big Data ? MB, GB,TB, PB Records Transactions Tables, Files Volume Batch Near-time Real-time Streams Velocity Structured Unstructured Semi-Structured All theAbove Variety Source: Celent The 4Vs of Big Data
  5. 5. 4/8/15   3   Data Everywhere Trends in Tech Salary Reaffirm Source: http://marketing.dice.com/pdf/Dice_TechSalarySurvey_2015.pdf
  6. 6. 4/8/15   4   “ ” If you think you can, or if you think you can’t, you are correct. – Henry Ford Question Your confidence level in current mobile automation architecture? Cost of Finding Bugs 0 20 40 60 80 100 120 140 160 Req Design Code UnitTesting Integration Testing System Testing Test Prod Cost
  7. 7. 4/8/15   5   How it Starts! Stage 1 •  Company needs mobile presence •  They hire Mobile Devs andTesters (usually manual) Stage 2 •  App becomes too complex to cover all the permutations via manual testing •  Company hiresAutomation Engineers (SDET) and are told to “automate everything”! Stage 3 •  Full-on effort to catch-up and automate all features •  SDET burnout! Creating a Plan Successful Automation Plan Device Lab Automation Framework Prioritize FeatureTest Cases Stress/ Performance/ Other Additional Testing
  8. 8. 4/8/15   6   Creating a Device Lab Creating a Device Lab (Using Big Data) Total # of Devices Devices with most # of reported bugs Your most Popular Devices Time box and add bug to your backlog Buy/Loan/ Rent device and bring it in- house
  9. 9. 4/8/15   7   Creating a Device Lab 30% 17% 13% 5% 4% 4% 3% 3% 3% 3% 2% 2% 2% 2% 2% 5% Apple LG MS770 Samsung Galaxy SIII Microsoft Coolpad Quatro 4G ZTE N9210 Samsung GalaxyAdmire 4G Droid RAZR Samsung Galaxy Note II LG Esteem LG MS870 SamsungAdmire Samsung Epic 4G Samsung Galaxy SII Samsung Omnia II Other Total # of Devices > 1850!! Pick an Automation Test Framework
  10. 10. 4/8/15   8   Prioritize KPI Customer Usage Data Finance (Revenue Stream) Data Marketing/ Social Data User Usage Pattern Home Screen, 40% 1 Detail Screen, 20% 2 Detail Screen, 15% 3 Detail Screen, 10% All OtherValues, 15%
  11. 11. 4/8/15   9   Tests Real User, Marketing and Finance Data Stress ServerVs. UI Data New Features Performance System Under Test Production data Test Results RunTests Quality Assessment Stress Testing — Find  Resource  Leaks   — Find  App’s  Capacity  and   Capabili:es   — Find  Memory  and   Ba>ery  Consump:on   Trends  
  12. 12. 4/8/15   10   Server Vs. UI Testing Server Client Test Framework •  Verify  data  is  in-­‐sync   during  tes:ng   •  Ensure  no  data  loss   during  test  progress   •  Detect  UI  TTL  (Time  to   Load)  on  devices   under  various   condi:ons   Performance Testing (Analyze and Record KPIs)
  13. 13. 4/8/15   11   Effective Testing Write Once,Test Anywhere Active Monitoring Test Re-Use Performance Availability Conclusion

×