This is a talk I gave at Software Testing Atlanta Conference (STAC 2015). ABSTRACT:: Learn about developing and maintaining automation tests for web and mobile applications. In this two part presentation, Shauvik will cover "Move Fast" - how to efficiently generate automation test scripts for manual test cases, and "Fix Tests" - strategies for maintaining tests between regressions or platforms.

1. MOVE FAST AND FIX TESTS
Shauvik Roy Choudhary
@shauvik http://shauvik.com
GOING FROM
MANUAL TO AUTOMATED:

4. MOVE FAST AND FIX TESTS
Shauvik Roy Choudhary
@shauvik http://shauvik.com
GOING FROM
MANUAL TO AUTOMATED:

5. MOVE FAST AND FIX TESTS
Shauvik Roy Choudhary
@shauvik http://shauvik.com
GOING FROM
MANUAL TO AUTOMATED:
Checks

6. ABOUT ME
• PhD Thesis:
Cross-platformTesting & Maintenance of
Web & Mobile Apps
• Industry:
Google,Yahoo!, FullStory, Fujitsu Labs, IBM
Research, HSBC, Goldman Sachs
• Entrepreneurship:
GeorgiaTechTI:GER program,VentureLab,
ATDC,TechSquare Labs
Shauvik Roy Choudhary
PhD, GeorgiaTech
Founder, CheckDroid

7. OUTLINE
• MOTIVATION: Why Automate ?
• MOVE FAST: How to Automate Quickly?
• FIX TESTS: How to Resolve Automation Issues?
• SCENARIO: Android UITesting
• DISCUSSION

8. WHY AUTOMATETESTS?
Software
ManualTesting

9. WHY AUTOMATETESTS?
Software
ManualTesting

10. WHY AUTOMATETESTS?
Software
ManualTesting

11. WHY AUTOMATETESTS?
Software
ManualTesting

12. WHY AUTOMATETESTS?
Software
ManualTesting

13. PLATFORMS
Web Tester

14. PLATFORMS
Web Tester

15. PLATFORMS
Web Tester

16. PLATFORMS
Web Tester

17. PLATFORMS
Web Tester

18. PLATFORMS
Web Tester

19. PLATFORMS
Web Tester

20. PLATFORMS
Web Tester

21. Mobile Tester
Lollipop
Android 5.0
9

22. HOWTO AUTOMATE?

23. HOWTO AUTOMATE?
WHAT
Parts of your App

24. HOWTO AUTOMATE?
HOW
Infrastructure
Tool Support
WHAT
Parts of your App

25. WHAT:TOTEST
• Functional Tests
Positive and negative scenarios
• Non-Functional Tests
Performance, Security, Compatibility…
• Previous Issues (identified Manually)
Never to occur in Regressions

26. HOW: INFRASTRUCTURE
• Unit Tests
xUnit for different programming languages
• UI Tests
Selenium/WebDriver for Web Apps
Appium for Mobile (iOS,Android)
Espresso for Android

27. IDEALTESTING PYRAMID
by Mike Cohn

28. ICE-CREAM CONE

29. HOUR GLASS
source: just-about.net

30. TESTING DIAMOND
Source: toddlittleweb.com

31. CHRISTMASTREE
source: just-about.net

32. UITESTS
CLOSESTTO
END-USERTESTING

33. MOVE FAST
How to automate quickly?

34. RELIABLE INFRASTRUCTURE

35. VISUALTEST RECORDING

36. FIX TESTS
How to fix broken tests?

37. UITESTING ISSUES

38. UITESTING ISSUES
• FRAGILITY

39. UITESTING ISSUES
• FRAGILITY
• SPEED

40. UITESTING ISSUES
• FRAGILITY
• SPEED
• FLAKINESS

41. 1. FRAGILETESTS

42. REDUCING FRAGILITY
• Use durable selectors
• IDs over XPath
• Relative XPaths over Absolute XPaths
• Other durable labels (class, content-desc..)

43. 2. SLOWTESTS

44. MAKINGTESTS FAST
• Don’t use Sleeps
• If absolutely necessary, sleep while polling
• Replace slow resources with stubs

45. doSomething()
Thread.sleep(1000)
checkSomething()

46. doSomething()
do{
Thread.sleep(1000)
}while(!loaded)
checkSomething()

47. FLAKINESS
UIThread
TestThread
Work in
Queue

48. FLAKINESS
UIThread
TestThread
Work in
Queue
click

49. FLAKINESS
UIThread
TestThread
Work in
Queue
click
Motion
down
Motion
up

50. FLAKINESS
UIThread
TestThread
Work in
Queue
click
Motion
down
Motion
up
assert

51. FLAKINESS
UIThread
TestThread
Work in
Queue
click
Motion
down
Motion
up
assert

52. FLAKINESS
UIThread
TestThread
Work in
Queue
click
Motion
down
Motion
up
assertsleep

53. FLAKINESS
UIThread
TestThread
Work in Queue
click
Motion
down
Motion
up
assertsleep

54. TESTS

55. 3. FLAKYTESTS
Image credits: Google, GTAC talk

56. REDUCING FLAKINESS
• Have a hermetic test environment
Source: googletesting.blogspot.com

57. REDUCING FLAKINESS
• Have a hermetic test environment
Source: googletesting.blogspot.com

58. FIX TESTS

59. 1. FRAGILETESTS
FIX TESTS

60. 1. FRAGILETESTS
2. SLOWTESTS
FIX TESTS

61. 1. FRAGILETESTS
2. SLOWTESTS
3. FLAKYTESTS
Image credits: Google, GTAC talk
FIX TESTS

62. ANDROID UI TESTING

63. MOBILETESTING PYRAMID
Source: Daniel Knott,AdventuresInQA
Author: Hand-On Mobile AppTesting

64. ANDROIDTESTINGTOOLS
Image credits: BitBar

65. ESPRESSO: HOW IT WORKS
UIThread
TestThread
Work in
Queue
Test Case
Motion
down
Motion
up
Espresso
action assert
Blocked

66. ESPRESSO METHODS
onView(Matcher)
.perform(ViewAction)
.check(ViewAssertion)

67. EXAMPLES
onView(withId(R.id.greeting))
.perform(click());
onView(withText("Hello Steve!"))
.check(matches(isDisplayed()));

68. EXAMPLES
onView(withId(R.id.greeting))
.perform(click());
onView(withText("Hello Steve!"))
.check(matches(isDisplayed()));
Find

69. EXAMPLES
onView(withId(R.id.greeting))
.perform(click());
onView(withText("Hello Steve!"))
.check(matches(isDisplayed()));
Find
Do Stuff

70. EXAMPLES
onView(withId(R.id.greeting))
.perform(click());
onView(withText("Hello Steve!"))
.check(matches(isDisplayed()));
Find
Do Stuff
Check

71. onView(Matcher<View>)
● withId
● withText
● withContentDescription
● isDisplayed
● hasFocus
● hasSibling
● ...
● mySpecialMatcher

72. perform(ViewAction)
● click
● longClick
● doubleClick
● typeText
● scrollTo
● ...
● myCustomAction

73. check(ViewAssertion)
● matches(...)
● doesNotExist
● myCustomAssertion

74. HIERARCHY
withParent(Matcher)
withChild(Matcher)
hasDescendant(Matcher)
isDescendantOfA(Matcher)
hasSibling(Matcher)
isRoot()
UI PROPERTIES
isDisplayed()
isCompletelyDisplayed()
isEnabled()
hasFocus()
isClickable()
isChecked()
isNotChecked()
withEffectiveVisibility(…)
isSelected()
ROOT MATCHERS
isFocusable()
isTouchable()
isDialog()
withDecorView(…)
isPlatformPopup()
COMMON HAMCREST
MATCHERS
allOf(Matchers)
anyOf(Matchers)
is(...)
not(...)
endsWith(String)
startsWith(String)
SEE ALSO
Preference matchers
Cursor matchers
USER PROPERTIES
withId(…)
withText(…)
withTagKey(…)
withTagValue(…)
hasContentDescription(…)
withContentDescription(…)
withHint(…)
withSpinnerText(…)
hasLinks()
hasEllipsizedText()
hasMultilineText()
INPUT
supportsInputMethods(…)
hasImeAction(…)
CLASS
isAssignableFrom(…)
withClassName(…)
Matchers
CLICK/PRESS
click()
doubleClick()
longClick()
pressBack()
pressImeActionButton()
pressKey([int/EspressoKey])
pressMenuKey()
closeSoftKeyboard()
openLink(…)
GESTURES
scrollTo()
swipeLeft()
swipeRight()
swipeUp()
swipeDown()
TEXT
clearText()
typeText(String)
typeTextIntoFocusedView(String)
replaceText(String)
POSITION ASSERTIONS
isLeftOf(Matcher)
isRightOf(Matcher)
isLeftAlignedWith(Matcher)
isRightAlignedWith(Matcher)
isAbove(Matcher)
isBelow(Matcher)
isBottomAlignedWith(Matcher)
isTopAlignedWith(Matcher)
LAYOUT ASSERTIONS
noEllipsizedText(Matcher)
noMultilineButtons()
noOverlaps([Matcher])
View Actions
matches(Matcher)
doesNotExist()
selectedDescendantsMatch(…)
View Assertions
onView(Matcher)
.perform(ViewAction)
.check(ViewAssertion)
CHEAT SHEET
2.0

75. BARISTA
EspressoTest Generation
http://checkdroid.com/barista

76. DEMO

77. Tests

78. TEST CLOUD

80. WHY AUTOMATETESTS?
Software
ManualTesting

81. WHY AUTOMATETESTS?
Software
ManualTesting
MOVE FAST
How to automate quickly?

82. WHY AUTOMATETESTS?
Software
ManualTesting
MOVE FAST
How to automate quickly?
1. FRAGILETESTS
2. SLOWTESTS
3. FLAKYTESTS
Image credits: Google, GTAC talk
FIX TESTS

83. WHY AUTOMATETESTS?
Software
ManualTesting
MOVE FAST
How to automate quickly?
1. FRAGILETESTS
2. SLOWTESTS
3. FLAKYTESTS
Image credits: Google, GTAC talk
FIX TESTS
ANDROID UI TESTING

84. DISCUSSION
Are you currently automating tests?
What are some challenges that you see?

85. THANKS!
Shauvik Roy Choudhary
@shauvik http://shauvik.com