1. Cucumber Acceptance
Tests
Lessons Learned

2. Cucumber Acceptance
Tests
Lessons Learned
A Frightening Journey Inside Cheezy’s Mind

3. Life is good

4. Starting Point

5. What should the
testers do?

6. Tester
Developer
Project Ping Pong

7. The true cost of a defect
Code Test

8. The true cost of a defect
Code Test

9. The true cost of a defect
Code Test

10. The true cost of a defect
Code Test

11. The true cost of a defect
Steps
Code Test

12. The true cost of a defect
Steps
Code Test

13. The true cost of a defect
Review /
Prioritize
Steps
Code Test

14. The true cost of a defect
Review /
Prioritize
Steps
Code Test

15. The true cost of a defect
Review /
Describe
Prioritize
Steps
Code Test

16. The true cost of a defect
Review /
Describe
Prioritize
Steps
Code Test

17. The true cost of a defect
Review /
Describe
Prioritize
Reproduce Steps
Code Test

18. The true cost of a defect
Review /
Describe
Prioritize
Reproduce Steps
Code Test
Update
Fix Defect Retest
Document

20. Capers Jones
Software Assessments, Benchmarks, and Best Practices
Addison-Wesley, 2000

21. What else?

22. What else?
• Unhappy Customers

23. What else?
• Unhappy Customers
• Demoralized Demotivated Team
• Team Flight

24. What else?
• Unhappy Customers
• Demoralized Demotivated Team
• Team Flight
• Pressure

25. QA Went Beyond Spec

27. Testing cut short
We have a date to make
so testing needs to end now!

28. Manual Testing (not ET)
1

29. Manual Testing (not ET)
1.5
1

30. Manual Testing (not ET)
2.2
1.5
1

31. Manual Testing (not ET)
6.4
5.3
4.0
3.0
2.2
1.5
1

32. Automation Fails
“Record / Playback testing tools should be clearly
labeled as training wheels” - Elizabeth Hendrickson

33. First attempts at automation
just automated manual tests

34. First attempts at automation
just automated manual tests
yielding brittle tests!

35. Little or No
Collaboration

36. 3 Letter Word

37. 3 Letter Word
4 Letter Word

38. Our testers are not very
good because we have way
too many bugs!

39. Hardening Phase

40. Fear

41. Bug List

42. “Cheezy”
Coach Extraordinaire
(with a dirty little secret)

43. Leave QA Alone

44. Testing was constraint

46. Tester
Mr. T

47. I can handle anything that
fool Cheezy can throw at me!

48. Thought Process?

49. Thought Process?
• Mr. T is very strong

50. Thought Process?
• Mr. T is very strong
• He can be quite intimidating

51. Thought Process?
• Mr. T is very strong
• He can be quite intimidating
• Did he intimidate the defects out of the
system?

52. Thought Process?
• Mr. T is very strong
• He can be quite intimidating
• Did he intimidate the defects out of the
system?
• Did the approach actually work?

53. The Dogs

55. Early attempts were not
always successful

56. Repeating Mistakes

57. Repeating Mistakes
Test
App

58. Repeating Mistakes
Test
Test
Test
Test
Test
Test
Test App
Test
Test
Test
Test
Test
Test

59. Repeating Mistakes
App

60. Repeating Mistakes
App

61. Works in the small

62. Large Project Fail

63. Hard to understand

64. Hard to understand

65. Communication?

66. Kinda makes you sad:(

67. Success at last!

68. What is it?

69. Acceptance Test Driven
Development

70. Acceptance Test Driven
Development
Behavior Driven
Development

71. Acceptance Test Driven
Development
Behavior Driven
Development
Executable Specifications

72. Acceptance Test Driven
Development
Behavior Driven
Development
Executable Specifications
Specification by Example

73. What did we learn?

74. The words friggin’ matter

75. Collaboration is the goal

76. Automation is necessary
(and easy)

77. Quality is a team sport

79. Getting the words right

80. Getting the words right
was the hardest part

85. Speak it!

86. Collaboration

87. We discovered walls
were still standing

88. Done Done is Evil

89. We Need Automation

90. We Need Automation
Started off thinking it was all about
automation

91. We Need Automation
Started off thinking it was all about
automation
Initially automated too much

92. Focus on Behavior
Conversion of manual scripts == FAIL

93. Brittle test

94. Brittle test

95. Brittle test

96. Engineering Principles

97. Engineering Principles
Test code is production code!

98. Engineering Principles
Test code is production code!
Keep it clean

99. Engineering Principles
Test code is production code!
Keep it clean Keep it D.R.Y.

100. Engineering Principles
Test code is production code!
Keep it clean Keep it D.R.Y.
Build abstractions

101. Clean D.R.Y. Test

102. Tools Goals

103. Tools Goals
Make it very easy to adopt and simple to use

104. Tools Goals
Make it very easy to adopt and simple to use
Provide “template” for code

105. Tools Goals
Make it very easy to adopt and simple to use
Provide “template” for code
Make it simple to update as app changes

106. Data?

107. Data?
Most data doesn’t matter

108. Data?
Most data doesn’t matter
Focus on what matters

109. Data?
Most data doesn’t matter
Focus on what matters
Provide defaults for everything else

110. Nice Picture

111. Still have a ways to go

112. Still have a ways to go
Test Data Management

113. Still have a ways to go
Test Data Management
Parallel Execution

114. Who should automate?

115. Who should automate?
Specification

116. Who should automate?
Specification Automation

117. Who should automate?
Specification Automation Application

118. Who should automate?
Specification Automation Application Exploration

119. Who should automate?
Specification Automation Application Exploration
Specification

120. Who should automate?
Specification Automation Application Exploration
Specification Application
Automation

121. Who should automate?
Specification Automation Application Exploration
Specification Application Exploration
Automation

122. Who should automate?
Specification Automation Application Exploration
Specification Application Exploration
Collaboration
Automation

123. Quality is driven by
everybody

124. How do defects get
into software?

125. How do we get them
out?

126. If you find defects after
development you are
already too late

128. IT’S TIME

129. TO CHANGE

130. THE WAY

131. WE WORK!

132. COLLABORATION

133. WINS!

134. Thank You