This document discusses test automation strategies for mobile applications. It begins with an introduction of the presenter, Kazuaki Matsuo. It then covers various aspects to consider when designing test automation, including scope, phase, level, size and type. For each aspect, it provides examples from Cookpad's test automation for their Android and iOS applications in both Japan and globally. It discusses adapting concepts to real products and challenges integrating performance testing. Finally, it lists some common tools used for mobile test automation.
2. Who I am
2
Kazuaki Matsuo
• International Business Development Product/Engineering Group
• Software Engineer in Quality
• Social Media
• Twitter: @Kazu_cocoa, GitHub: KazuCocoa
• OSS: Appium Project
4. Okay, let’s go
4
Motivations
• Build fast feedback cycle
• Make test deterministic
• Design and build product lines
• Build “Culture of automation” to be autonomous organisation
5. By the way,
5
Can you work with the same context?
A: What about this integration test?
B: Well… we should use mock to make them stable.
A: I won’t use mocking since it hides real world.
B: I see, but without mock, the test may be flaky.
A: This is integration and my understanding of the type is less
mocking and close to in the wild, though…
6. In this session, you can learn
6
Concepts and Ways you go
• Concepts for automated test design / plan
• How adapt them to real projects/products
• Cookpad International applications
• Cookpad Japan applications
7. In this session, you can learn
7
Concepts and Ways you go
• Concepts for automated test design / plan
• How adapt them to real projects/products
• Cookpad International applications
• Cookpad Japan applications
Won’t talk about
business/humans/
techniques/tools
33. What phase would you like to automate?
33
unit test
component
test
coding
low-level
design
integration
test
high-level
design
system
test
acceptance
test
specification
analyse
requirement
34. What phase would you like to automate?
34
unit test
component
test
coding
low-level
design
integration
test
high-level
design
system
test
acceptance
test
specification
analyse
requirement
42. What activity is the target?
42
Plan/Design Implement Run
Analyse
The Results
43. How deep would you like to drill down?
43
https://blog.appdiff.com/test-autonomy-levels-7de7967d030e
0. Manual Testing
1. Scripted Testing
2. Exploratory Bots
3. Human-Directed
4. Generative
5. Fully Autonomous
60. Before stepping into concrete design…
60
Define your ideal product line
• What is your team’s ideal product line?
• To achieve the line, what kind of activities would you automate?
• Planning/Designing test automation as well
80. Type
80
Example
A/B Tests
Story Tests (Written First)
UX (User Experience) Testing
Prototypes (paper/wireframes)
Simulations
Unit Tests
Component Tests (code level)
Testing Connectivity
Exploratory Testing
Workflows
System Integration
(business oriented)
Usability Testing
User Acceptance Testing
Performance Testing
Load Testing
Security Testing
Quality Attributes (…ilities)
Outside-in
Barrier-free
Collaborative
Business Facing
Technology Facing
GuideDevelopment
CritiquetheProduct
http://www.informit.com/articles/article.aspx?p=2253544&ranMID=24808
81. Type
81
Example
Story Tests (Written First)
Unit Tests
Component Tests (code level)
Testing Connectivity
User Acceptance Testing
Business Facing
Technology Facing
GuideDevelopment
CritiquetheProduct
http://www.informit.com/articles/article.aspx?p=2253544&ranMID=24808
88. History
88
Cookpad Android / iOS run long history
following with Android / iOS platforms evolution
https://academy.realm.io/posts/tasting-tests-at-cookpad-try-swift-2017/
89. Current status ( Sep?, 2014~ Feb, 2018)
89http://techlife.cookpad.com/entry/2016/08/13/test-size-for-mobile
L: UI Component based
M: Integration Tests
S: Unit Tests
E: User Scenario based
90. 90
L: UI Component based
M: Integration Tests
S: Unit Tests
E: User Scenario based
http://techlife.cookpad.com/entry/2016/08/13/test-size-for-mobile
some of manual test
for regressions
Current status ( Sep?, 2014~ Feb, 2018)
92. Any plan or design?
92
No particular entire plan / design
• but, one interesting automaton
93. Type
93
Example
A/B Tests
Story Tests (Written First)
UX (User Experience) Testing
Prototypes (paper/wireframes)
Simulations
Unit Tests
Component Tests (code level)
Testing Connectivity
Exploratory Testing
Workflows
System Integration
(business oriented)
Usability Testing
User Acceptance Testing
Performance Testing
Load Testing
Security Testing
Quality Attributes (…ilities)
Outside-in
Barrier-free
Collaborative
Business Facing
Technology Facing
GuideDevelopment
CritiquetheProduct
http://www.informit.com/articles/article.aspx?p=2253544&ranMID=24808
94. Type for Android
94
Example
Story Tests (Written First)
Unit Tests
Component Tests (code level)
Testing Connectivity
User Acceptance Testing
Business Facing
Technology Facing
GuideDevelopment
CritiquetheProduct
http://www.informit.com/articles/article.aspx?p=2253544&ranMID=24808