Taking lessons from the classic writings on Software Testing, we'll walk through what are industry tried and true approaches for developing a robust and effective test suite. Many of the Testing philosophies and strategies today have their origins as far back as the 60's, and really got their legs during the advent of "Extreme Programming" and other early "Agile" methodologies from the 1990s. In this talk we'll take many of these tried and true test strategies and show how one would apply them in a real way to an Ember project.

1. Developing an Ember Test
Strategy
Todd Jordan

2. About Me

3. About Me

4. About Me

5. About Me

6. About Me
@tddjordan

7. ❤

8. Testing in Ember
The Hidden Levels

9. With Some Origin Stories Mixed In

10. Speed and Feedback Loops

11. Speed and Feedback Loops

12. https://deming.org/deming/timeline

13. Act
Check Do
Plan
Deming
Circle

15. https://blog.deming.org/2016/10/toyotas-management-history/

16. https://www.amazon.com/Implementing-Lean-Software-Development-Concept/dp/0321437381/

17. Boehm’s Law
0
$
$$
$$$
Requirments Design Development Test Prod
https://docs.google.com/viewer?url=https%3A%2F%2Fwww.cs.umd.edu%2Fprojects%2FSoftEng%2FESEG%2Fpapers%2F82.78.pdf

18. The faster your feedback, the more
successful you’ll be

19. The faster your feedback, the more
successful you’ll be

20. Test Types
What Tests to Write and When to Use Them
https://www.flickr.com/photos/chiffheed/6889521843

21. http://wiki.c2.com/?WardAndKent

22. https://agilemanifesto.org

23. Business Facing
Technology Facing
SupportProgramming
CritiqueProduct
Performance
Security
Usability
Exploratory
Acceptance
End To End
Rendering
Unit
http://www.exampler.com/old-blog/2003/08/21.1.html#agile-testing-project-1
https://www.amazon.com/Agile-Testing-Practical-Guide-Testers/dp/0321534468

24. Business Facing
Technology Facing
SupportProgramming
CritiqueProduct
Performance
Security
Usability
Exploratory
Acceptance
End To End
Rendering
Unit
http://www.exampler.com/old-blog/2003/08/21.1.html#agile-testing-project-1
https://www.amazon.com/Agile-Testing-Practical-Guide-Testers/dp/0321534468

25. UI
Service
Unit
fast
slowpricey
cheap
https://www.amazon.com/Succeeding-Agile-Software-Development-Using/dp/0321579364
https://martinfowler.com/bliki/TestPyramid.html

26. Ember Testing
Ember Unit Tests - Call functions directly
(10-100ms)
Ember Rendering Tests - Call components via the
template compiler (200-600 ms)
Ember Application Tests - Interact with the entire
Ember application through the browser
(700-3000ms with stubbed HTTP)

27. 1000 Tests
Unit
Rendering
Application
Minutes
0 10 20 30 40
33.3
6.7
1.7
Assumes 100ms Unit, 400ms Rendering, 2000ms Application

28. 1000 tests: Distributed
Minutes
0
3.5
7
10.5
14
33/33/33 40/40/20 40/50/10
Unit Rendering Application

29. * With Doubles
Application*
Unit/Rendering
E2E
Service

30. Test Workflow
A.K.A a nondogmatic discussion of Test First

33. “The original description of TDD was in an
ancient book about programming.
It said you take the input tape, manually
type in the output tape you expect,
then program until the actual output tape
matches the expected output. ”
-Kent Beck: OG Extreme Programmer, TDD Rediscoverer

35. TDD
Write a Failing Test
Make it Pass
Refactor
Repeat!!

36. Look Familiar?
Test 1 … Test n
Works!

37. Look Familiar?
Test 1 … Test n
Works!

38. -Everyone at some point, probably
“Won’t working this way slow you down?”

39. Code
http://geepawhill.org/tdd-and-the-lump-of-coding-fallacy/

40. Code
http://geepawhill.org/tdd-and-the-lump-of-coding-fallacy/

41. Typing

42. Typing

43. Typing
Waiting on NPM

44. Typing
Waiting on NPMReading Code

45. Typing
Waiting on NPM
Googling
Reading Code

46. Typing
Waiting on NPM
Googling
Reading Code
Chatting

47. Typing
Waiting on NPM
Googling
Reading Code
Chatting
Running Stuff

48. Typing
Debugging
Waiting on NPM
Googling
Reading Code
Chatting
Running Stuff

49. Typing
Debugging
Waiting on NPM
Googling
Reading Code
Chatting
Running Stuff
Automated Tests

50. Typing
Debugging
Waiting on NPM
Googling
Reading Code
Chatting
Running Stuff
Automated Tests

51. Typing
Debugging
Waiting on NPM
Googling
Reading Code
Chatting
Running Stuff
Automated Tests

52. Classic TDD
API
https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530

53. Classic TDD
Test
API
https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530

54. Classic TDD
Test
API
https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530

55. Classic TDD
TestTest
API
https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530

56. Classic TDD
TestTest
API
https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530

57. Classic TDD
Test
Your object tree forms as part of
your refactor step
Test
API
https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530

58. Example in Ember: A Page Component
Last
Email
Submit
First
Input: Object with First, Last,
Email, Profile Pic
Input: Save action
Output: Rendered DOM
Output: Action invoked
Upload

59. Classic TDD
API

60. Classic TDD
Test
API

61. Classic TDD
Test
Profile Page
API

62. Classic TDD
Test
Profile Page
API
Test

63. Classic TDD
Test
Profile Page
API
Test
Profile Picture

64. Classic TDD
Test
Profile Page
API
Test
Profile Picture

65. Macro Inside Out

66. Macro Inside Out
Stubs
Tests
Core Logic

67. Data
Macro Inside Out
Stubs
Tests
Core Logic

68. Data
Macro Inside Out
Tests
API
Stubs
Tests
Core Logic

69. Data
UI
E2E
Macro Inside Out
Tests
API
Stubs
Tests
Core Logic

70. TDD with Mock Objects
Mocks assert the behavior
of your object’s
collaborators
Enable a form of “Need
Driven Development”
See sinon.js and ember-
sinon
https://www.amazon.com/Growing-Object-Oriented-Software-Guided-Tests/dp/0321503627
https://docs.google.com/viewer?url=http%3A%2F%2Fjmock.org%2Foopsla2004.pdf

71. TDD with Mocks
http://www.natpryce.com/articles/000342.html
API

72. TDD with Mocks
http://www.natpryce.com/articles/000342.html
Test
API

73. TDD with Mocks
http://www.natpryce.com/articles/000342.html
Test
API

74. TDD with Mocks
http://www.natpryce.com/articles/000342.html
Test
Mock Mock
API

75. TDD with Mocks
http://www.natpryce.com/articles/000342.html
Test
Mock Mock
API
Test

76. TDD with Mocks
http://www.natpryce.com/articles/000342.html
Test
Mock Mock
API
Test
Mocks are your roles

77. TDD with Mocks
http://www.natpryce.com/articles/000342.html
Test
Mock Mock
API
Test
Your mocks become the APIs
for your next test
Mocks are your roles

78. TDD with Mocks
http://www.natpryce.com/articles/000342.html
Test
Mock Mock
API
Test
Your mocks become the APIs
for your next test
Keep going until you reach a
“leaf”, or an external
boundary
Mocks are your roles

79. Example in Ember: Chat Service
Hi!
New phone,
who dis?
todd.jordan
locks
1:00pm
1:05pm
Fetches chat history
Receives incoming messages in
realtime
Sends outgoing messages

80. TDD with Mocks
API

81. TDD with Mocks
Test
API

82. TDD with Mocks
Test
Chat Service
API

83. TDD with Mocks
Test
Chat Service
API
Realtime Service
Receive Incoming
Profile Service
Fetch Profile Info
History Service
Past Chats

84. TDD with Mocks
Test
Chat Service
Test
API
Realtime Service
Receive Incoming
Profile Service
Fetch Profile Info
History Service
Past Chats

85. Macro Outside In (Need Driven Development)

86. E2E
Macro Outside In (Need Driven Development)

87. E2E
Macro Outside In (Need Driven Development)
UI
Mocks
Tests

88. E2E
Macro Outside In (Need Driven Development)
Tests
API
Mocks UI
Mocks
Tests

89. E2E
Tests
Mocks
Core Logic
Macro Outside In (Need Driven Development)
Tests
API
Mocks UI
Mocks
Tests

90. E2E
Tests
Mocks
Core Logic
Macro Outside In (Need Driven Development)
Tests
API
Mocks
Data
UI
Mocks
Tests

91. E2E
Tests
Mocks
Core Logic
Macro Outside In (Need Driven Development)
Tests
API
Mocks
Data API
UI
UI
Mocks
Tests

92. E2E
Tests
Mocks
Core Logic
Macro Outside In (Need Driven Development)
Tests
API
Mocks
Data API
UI
UI
Mocks
Tests
*Done Iteratively, feature by feature

93. So… Which Should I do?

94. So… Which Should I do?
¯_(ツ)_/¯

95. - Nat Pryce, co-author: JMock, GOOS
“Mock objects are just a tool. They are one of many
tools that you need to use when doing TDD. Like any
tool, they are designed to help solve a set of
problems in a specific context.”
“So, instead of treating mock objects, or any other
technique or tool, as an all-or-nothing "golden
hammer", it's more helpful to understand where that
tool fits into the broader picture of TDD practice.”
https://groups.yahoo.com/neo/groups/testdrivendevelopment/conversations/messages/29686

96. Some Parting Perspectives

97. Do I Have To?
https://giphy.com/gifs/star-trek-riker-tng-tYelSVzQPtLdm

98. Do I Have To?
https://giphy.com/gifs/star-trek-riker-tng-tYelSVzQPtLdm

99. The Flawed Theory Behind Unit Testing
Classical TDD
Mockist TDD
Clean Room
Design By Contract
Michael Feathers
https://michaelfeathers.typepad.com/michael_feathers_blog/2008/06/the-flawed-theo.html

100. The Flawed Theory Behind Unit Testing
Classical TDD
Mockist TDD
Clean Room
Design By Contract
Michael Feathers
https://michaelfeathers.typepad.com/michael_feathers_blog/2008/06/the-flawed-theo.html
All

101. –Michael Feathers
“Quality is a function of thought and
reflection - precise thought and reflection.
That’s the magic. Techniques which
reinforce that discipline invariably increase
quality.”

102. Keep an eye on your feedback loops
Test in a methodical way that drives design
Thought and discipline are faster than quickly
spitting out code

103. Thanks!!!
@tddjordan
toddjordan
todd.jordan