Presented at All Things Open 2018 Presented by Dmitry Vinnik with Salesforce 10/22/18 - 11:30 AM - Mobile Track

1. Uphill Battle Of Mobile
Visual Regression
Dmitry Vinnik, @DmitryVinnik
Lead Software Engineer, Salesforce

2. Goals
Visual Testing (VT):
What, Why, How and When
New Perspective on Test Pyramid
Find VT Tool For Your Team
What are our goals?
2

3. Target Platforms
iOS
Android
Responsive
Apps
Hybrid
Apps
Target Platforms?
3

4. Let’s see some dataBut “Mobile” is so broad
Android
iOS
4

5. From the Top Level
5
How do we organize
QA Effort?

6. Agile Testing Quadrants
6
Testing Quadrants
From “Agile Testing: A Practical Guide for Testers and Agile Teams”
by Lisa Crispin and Janet Gregory

7. How do we usually test?
7
Let’s go in more details

8. Test Pyramid
↑ Fidelity
↓ Speed
↑ Cost
↓ Fidelity
↑ Speed
↓ Cost 8

9. Let’s try get more practical
9

10. Is anyone thirsty?
10

11. Test Pyramid & Drinking Glass
Pour Drink
Unit Integration E2E
Drinking Glass
11

12. Are we done?Not really...
12

13. Unit Integration E2E
Pour Drink
Test Pyramid & Drinking GlassDrinking Glass
13

14. What are we missing?
Same
Functionally
Different
Visually
UX
UX
What are we missing?
14

15. Visual Testing
Visual
Content
Page
Layout
Visual
Animation
Responsive
Design 15

16. Functional
Manual Testing
Visual
Manual Testing
16
is Difficultis Nearly
Impossible

17. A Little Bit of Science
17
A Little Bit of Science

18. Visual Testing
requires Automation
Conclusion?
18

19. But!
Where does it fit on
Test Pyramid?
19

20. On Every Level!
20

21. Test Pyramid
↑ Fidelity
↓ Speed
↑ Cost
↓ Fidelity
↑ Speed
↓ Cost 21

22. Remember the Title?
22
Uphill Battle Of Mobile
Visual Regression
Uphill Battle

23. 23
Test
Pyramid
Test Hill
Test
Mountain

24. Visual Test Mountain
↓ Fidelity
↑ Speed
↓ Cost
↑ Fidelity
↓ Speed
↑ Cost
24
Visual Test Mountain

25. Let’s Start Our Climb!
25

26. CSS/DOM Tests
26

27. Testing Demos
Responsive
App
Hybrid App
Android iOSAndroid iOS
27

28. Demo Time:
Snapshot Testing
28

29. Is it enough?No!Let me show you why
29

30. Quick Glance:
Responsive Design
in Action
30

31. Solution?
31

32. Visual Component Testing
32

33. Let’s Step Back First
33

34. Component Driven
Development
Reusable
Composable
Encapsulated
Semantic
34

35. CDD in Practice
35

36. Storybook
Library
Testing
Ground
Styleguide IDE
36

37. Quick Glance:
Storybook
37

38. Visual Component Testing
38

39. Testing Demos
Responsive
App
Hybrid App
Android iOS
39

40. Screenshot Testing
Action
Driver
View
Loader
Comparison
Driver 40

41. PhantomCSS
41

42. Demo Time:
PhantomCSS
42

43. BackstopJS
43

44. BackstopJS
44

45. Screenshot Testing:
Native
Action
Driver
View
Loader
Comparison
Driver 45

46. Screenshot Testing:
Native
Android screenshot-tests-for-android
iOS ios-snapshot-test-caseiOS ios-snapshot-test-case
46

47. Demo Time:
Screenshot Testing
Android
47

48. Enterprise
Screener.io
Flawlessapp.io
Chromatic
Percy.io
48

49. Best Practices
Screenshot
Names
Component
Testing First
Git Media
Standard
Test Pyramid 49

50. Summary
DOM/CSS
Not Enough
Dev = Test
Choose
Solution
Build
Solution 50

51. Page Visual Tests
51

52. Responsive Testing
Window Size
Transition
States
DOM/CSS
Screenshots
52

53. Demo Time:
Responsive Testing
with Selenium
53

54. Demo Time:
Native Visual Testing
with
Appium & Applitools
54

55. Enterprise
Applitools BrowserStack
SauceLabs TestObject
55
SauceLabs

56. Summit
You+
56

57. Call For Action
Start the Climb Visual Unit
Tests
Visual
Component
Tests
Visual Page
Tests
57

58. Q/A
58

59. About Speaker
Twitter: @DmitryVinnik
LinkedIn: in/dmitry-vinnik/
Email: dmitry@dvinnik.com
59