Contenu connexe Similaire à Mobile Apps Validation: Facing the Challenge (20) Mobile Apps Validation: Facing the Challenge3. Story of a transition
Investigation Prototyping Setup Execution
- Transition to a new platform
- Deep change in mind-set and culture
- Very short time to market (6 months)
- Pivotal role of QA
3 © Nokia 2011
4. Early investigation
Investigation Prototyping Setup Execution
- Quick “know-how injection”
- 3 to 4 team members (1 QA)
- Find out what we have and what we need
- Unit Test framework and CM tools evaluation
- Explore, try and adapt solutions
4 © Nokia 2011
5. Proof of concept
Investigation Prototyping Setup Execution
- ~Daily releases
- Unit Test, CM, CI and bug tracking tools in place
- Component tests evaluation
- UI Visual Test Automation evaluation
- Show some solutions are valuable and why
5 © Nokia 2011
6. Team set-up
Investigation Prototyping Setup Execution
- ~14 people (~35% QA), 2 Scrum Teams, 2-week sprints
- Ramp-up and knowledge transfer
- New CI and bug tracking tools in place
- “Feature branching”
- Let QA influence the architecture to increase testability
6 © Nokia 2011
7. Deliver!
Investigation Prototyping Setup Execution
- Fortnightly releases
- New UI Test Automation in place
- Exploratory+Automated testing
- “Master branch focused”
- Keep the master clean and working
7 © Nokia 2011
8. An adaptive strategy
- Different QA solutions across the project phases
- Work both on emulators and devices
Test coverage
120
100
80
60
40
20
0
Prototyping (I) Prototyping(II) Production (I) Production (II)
Unit Tests Component tests UI Automation Manual Testing
8 © Nokia 2011
9. Flexibility: Test granularity
UI Automated • Functional Test
Tests • Stress Tests
Component • UI native controls
Tests • UI custom controls
• Algorithmic failures
Unit Tests • Static libraries
9 © Nokia 2011
10. Adaptive: Test impact
- Low level services
- Single control and control-to-control interaction
- Complete app stack
UI Automated
test
UI layer
Control1 Control2
Component
test
Data Service Service
component Agent 1 Agent 2 Unit test
10 © Nokia 2011
11. Unit tests
- Constant sanity check on prototype
- Continuous low level verification on product
- Focused on the Model
• White box testing
• Use 3rd party (quasi-)stable frameworks
• Handle asynchronous calls scenarios
• Don’t push it too far!
11 © Nokia 2011
12. Component tests
- Efficient vertical testing of the application
- Focused on Model, View, ViewModel of single
Controls
12 © Nokia 2011
Interaction Properties APIs
13. UI automated tests
- Automated functional tests
- Helps stabilizing the product at each release
- Focused on View and ViewModel of the App
• Grey-to-black box testing
• Too easy means useless
• Let developers and testers share the same environment
13 © Nokia 2011
14. Conclusions
Investigation Prototyping Setup Execution
120
100
80
60
40
20
0
Prototyping (I) Prototyping(II) Production (I) Production (II)
Unit Tests Component tests UI Automation Manual Testing
14 © Nokia 2011
15. What we learned
• Guarantee a minimum quality level
−QA can only lead the change through a robust credibility
• Automate as much as you can
−Exploratory testing will cover edge cases and new scenarios
• Coding tests is the best test
−Testers and developers should share the same development
environment
15 © Nokia 2011
16. Improvements
• Component tests partially surrogate back-end mocking
• Create an automated test per each relevant bug
• CI must be continuous and continuously online
• Estimate the impact of a story change during the sprint
16 © Nokia 2011
17. We are hiring!
- Nokia in Berlin has several open positions
- These are exciting times!
www.nokia.com/careers
developer.jobs.berlin@nokia.com
17 © Nokia 2011