Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Build Your Mobile App Quality and Test Strategy
Similaire à Build Your Mobile App Quality and Test Strategy (20)
Plus de TechWell
Plus de TechWell (20)
Dernier
Dernier (20)
Build Your Mobile App Quality and Test Strategy
- 1. ME Half-day Tutorials Monday, April 30th, 2018 8:30 AM Build Your Mobile App Quality and Test Strategy Presented by: Jason Arbon Appdiff, Inc. Brought to you by: 350 Corporate Way, Suite 400, Orange Park, FL 32073 888---268---8770 ·· 904---278---0524 - info@techwell.com - http://www.stareast.techwell.com/
- 2. Jason Arbon Appdiff, Inc. Jason Arbon is the CEO of Appdiff, which is redefining how enterprises develop, test, and ship mobile apps with zero code and zero setup required. He was formerly the director of engineering and product at Applause.com/uTest.com, where he led product strategy to deliver crowdsourced testing via more than 250,000 community members and created the app store data analytics service. Jason previously held engineering leadership roles at Google and Microsoft and coauthored How Google Tests Software and App Quality: Secrets for Agile App Teams.
- 3. 4/17/17 1 App Quality and Testing Build Your Strategy Jason Arbon, CEO Appdiff.com
- 6. 4/17/17 4 Test Engineer Developer Dir of Product/Eng Founder/CEO
- 7. 4/17/17 5 How Google Tests Software App Quality
- 8. 4/17/17 6 Context Millions of App Store reviews Chats with Top App Teams Automating 1000’s of Apps Building Apps
- 9. 4/17/17 7
- 11. 4/17/17 9 Standardization Public User Feedback Per-App Cross Metrics Categories Expectations Structure ~45 Minute Sessions 25 min Learning 15 min Practicing QnA As long as folks like
- 12. 4/17/17 10 Topics Monsters Attributes Pillars and Actions Performance Automation Toolbelt Putting it all Together Surprise Inside : Test Strategy Document
- 13. 4/17/17 11 LARP Be a real app Tester Break into teams of 2-5
- 15. 4/17/17 13 Offline? Larp: Pick app and Read up
- 17. 4/17/17 1 App Quality and Testing Monsters Jason Arbon, CEO Appdiff.com Mobile Monsters
- 18. 4/17/17 2 Mobile is Different Apps are Different
- 19. 4/17/17 3 Monsters Top Sources of Mobile App Issues Users No Targeting On the go Expectations
- 21. 4/17/17 5 Reviews Reflect on you Public Can’t please all Agile/CI/CD No Time Too many builds Tech Debt
- 24. 4/17/17 8 Competition Many Avoid Steal LARP: Monsters Pick Top 3 Monsters for Your App Pick Top 2 Mitigations
- 26. 4/17/17 1 App Quality and Testing Attributes Jason Arbon, CEO Appdiff.com Mobile Attributes
- 27. 4/17/17 2 Attributes What Users Care About Source Analysis of 200M+ Reviews 1M+ Bugs 10K+ Test Cycles
- 28. 4/17/17 3 Elegance “Design is Garbage” Usability “Difficult to Navigate”
- 29. 4/17/17 4 Stability “Crashes with new Update!” Privacy “Will not remove financial information”
- 30. 4/17/17 5 Security “Can’t login anymore” Performance “Slow and frustrating”
- 31. 4/17/17 6 Satisfaction “Not cool anymore” Interop “Won’t Connect…”
- 32. 4/17/17 7 Content “Bots and fake profiles” Pricing “Charged for driver who didn’t show!”
- 33. 4/17/17 8 What to Focus on? What to Focus on?
- 34. 4/17/17 9 For your App’s Category? What to Focus on?
- 35. 4/17/17 10 What to Focus on? LARP: Attributes Pick Top 3 Attributes for Your App Pick Top 2 Mitigations
- 37. 4/17/17 1 App Quality and Testing Pillars and Actions Jason Arbon, CEO Appdiff.com Soft Science of Quality
- 38. 4/17/17 2 Pillar: Users Software Exists to Serve Users
- 40. 4/17/17 4 Regression Define Execute Respond maintain Exploratory Define tours Explore Share Findings
- 41. 4/17/17 5 Automation Identify Build Respond Maintain Pillar: Data Objective Insight into Quality and Value
- 44. 4/17/17 8 Projection Metrics Low/High Key Predictions Share Pillar: Biz Why invest in quality?
- 45. 4/17/17 9 Biz: Actions KPIs Competitive E2E Cost and speed KPIs Relate Test to KPIs Measure Trend Impact
- 47. 4/17/17 11 Cost + Speed Features Processes Grease Tooling Pillar: Team Team is Everything Team does Everything Nothing happens without the team
- 49. 4/17/17 13 Learn Cross-train Technical Customer visit Support Visibility Kudos (public) Kudos (private) Dashboard Monetary
- 50. 4/17/17 14 Reward 20% Gift Certificates Raise/Promo Opportunities LARP: Most Important Pillars Pick Top 2 Pillars Pick Top 2 Actions
- 52. App Quality and Testing Application Performance Monitoring Jason Arbon, CEO Appdiff.com
- 53. Production APM
- 54. APM Server Performance Monitoring In-App Performance Monitoring Crash Collection Engagement/Usage
- 56. SDKs Install SDK + Code: App and Server Maintain/Update Can cause crashes and perf issues themselves Several in same app can conflict
- 57. AppDynamics
- 58. Twitter Fabric
- 59. New Relic
- 60. Runscope
- 61. Limitations of Production APM Issues found *after* user impact Difficult to interpret graphs, charts, dials Measure API and Code--Not User Perception
- 63. Why Pre-Production, User-Focused APM? Catch issues *before* user impact Measure user-perceived performance Zero-Code, Zero SDK. With CI, know which code change impacted performance
- 64. Appdiff: Automatically walk app as user
- 66. Performance Perf Feature Area
- 67. Appdiff: Trends Over Time
- 68. End User Production Monitoring
- 69. LARP: Automation Decide whether to use APM Which APMs?
- 71. 4/17/17 1 App Quality and Testing Automation Jason Arbon, CEO Appdiff.com Many Try Many Fail ..and try again
- 72. 4/17/17 2 Overall Some teams (~30%) have unit tests (run in <1 minute) Few teams build and maintain UX-based tests Biggest App teams often have <50 automated tests Talent Unicorns: Hard to Find Developer+Tester Tornado: Harder to retain as they move into Development roles Expensive!
- 73. 4/17/17 3 Automation Breaks when app Changes Record and Playback? Demos well False Hope Brittle Doesn’t scale Often Proprietary
- 74. 4/17/17 4 I was warned and accept responsibility for trying because it sounds cool. Current Generation
- 75. 4/17/17 5 Calabash Cross Platform Mobile only Ruby-only Cucumber More Layers Debugging Not for Testing!
- 76. 4/17/17 6 Creator Doesn’t Want you To Use it! Cucumber reached a million downloads in the first three years and 5 million downloads 3 years later. I'm happy to have created such a popular tool, but saddened to see how it's misused and misunderstood. If you think Cucumber is a testing tool, please read on, because you are wrong. Espresso Android Only New/Fresh Sync with UX
- 77. 4/17/17 7 Appium Cross Platform Web Too! Open Any language Still Fragile Test Execution
- 78. 4/17/17 8 Devices vs Emulators Expensive/Painful Scaling issues (even cloud) Sleep well at night Finally ‘fast enough’ Almost Functionally Equivalent Cheap Scales Your own device lab
- 79. 4/17/17 9 Your own device lab Feel “in control” Scratch As a Service Cloud Device Lab
- 80. 4/17/17 10 Cloud Devices via API Cloud Simulators
- 81. 4/17/17 11 Device Matrix Usage and Crash Data Will Tell you Don’t Stress it Reviews What to Automate? Machines: Good at Quantitative Human: Good at Qualitative Don’t just automate manual test cases
- 82. 4/17/17 12 Next Generation Test Development and Execution as a service Type Test Cases into their Web UX They write and Run the tests for you Not mobile today, but soon?
- 83. 4/17/17 13 Still a Fundamental Problem
- 84. 4/17/17 14 Automated App Automation No Code. No Setup. Auto-Crawl Focus on what changes build to build UX + Performance Appdiff: Automatically walk app as user
- 85. 4/17/17 15 Appdiff: Auto-Automation Coverage Appdiff: Auto-Automation Coverage
- 86. 4/17/17 16 Appdiff: Auto-Automation Coverage Appdiff: Test Results
- 87. 4/17/17 17 LARP: Automation Automate or Not? Cloud or Local? Devices or Emulators? jason@appdiff.com Next Topic: Toolbelt
- 88. App Quality and Testing Toolbelt Jason Arbon, CEO Appdiff.com
- 89. Other Recommended Tools and Infrastructure
- 90. Build
- 91. BuddyBuild
- 92. CircleCI
- 93. Beta Management
- 94. AppSee
- 95. BuddyBuild
- 99. AppBot
- 100. User Feedback
- 101. Apptentive
- 103. Mixpanel
- 104. Heap
- 105. Crowd/Panel Testing
- 106. Applause
- 107. UserTesting.com
- 108. LARP: Toolbelt Chose 2 Tools
- 110. 4/17/17 1 App Quality and Testing Stitching Strategy Jason Arbon, CEO Appdiff.com Putting it all together
- 111. 4/17/17 2 A Balanced App Test/Quality Strategy Address several Monsters Address several Attributes Address several Pillars and Actions Pick pre and post production APM solution Pick Automation Strategy/Tools Pick 2 General Quality Tools Decide on Testing Cadence Testing Cadence Continuous Integration? Agile? Keys: ● Focus on Changes ● Rolling Passes ● Automation ● Metrics
- 112. 4/17/17 3 You have a Test Strategy now jason@appdiff.com linkedin.com/in/jasonarbon signup@ appdiff.com/mdt