Testing value first before correctness.

1. Test value first
before correctness
27-28 February 2017
LONDON
Laurent Py
@py_laurent
laurent.py@hiptest.net
https://hiptest.net

2. PLAN
2
The match: quality VS speed
Step 1: Testing the idea
Step 2: Testing first increment in production
Step 3: General availability of the feature
Benefits of a progressive approach to quality

3. 27-28 February 2017
LONDON
QUALITY VS SPEED

4. WE HAVE SPEED
4
Between 5 to 10
deployments/day
Our
DevOps
team
Devs/testers
Ops
Product manager
Users
Marketing
Support
4

5. WHAT ABOUT THE USER?
5
Speed of development feedback and learning
Value VS Quality
5

6. 27-28 February 2017
LONDON
STEP 1: TESTING THE IDEA

7. 8
• Use personae
• Capture the expected behavior with BDD
• Define business assumptions
Benefits
Support
Devs/testeurs
Marketing
Ops
Product
manager
WHAT VALUE DO WE WANT TO DELIVER?
8

8. 9
• Create a shared understanding with examples
• Based on a common business language
• Definition of stop
• When tests are automated → living documentation
BDD AT A GLANCE
9

9. 1111

10. 27-28 February 2017
LONDON
STEP 2: TESTING FIRST INCREMENT IN
PRODUCTION

11. 13
1. After step 1, we have our definition of success
2. We develop a first increment, do ET
3. We deploy it in production
FEATURE IS DEVELOPED
13

12. 14
• Based on usage
- NB users, clicks
- NB requests…
• Impact on
- Sign-up
- Revenue
- Retention…
LET’S MEASURE THE IMPACT
14

13. 15
EXAMPLES OF METRICS
15
Notification feature
• Usage
- NB clicks NB of notifications
- NB of users
• Business: none
Test refactoring
• Usage
- NB of refactoring actions
- NB of impacts
• Business: Retention

14. 16
LIVE CHAT TO GET REAL TIME FEEDBACK
16

15. 1717

16. 27-28 February 2017
LONDON
STEP 3: FEATURE FULLY AVAILABLE

17. 19
1. The impact has been validated
2. Investment on performance and non regression
(checks are fully automated)
3. Feature made available to all users with a switch
FEATURE BECOMES FULLY AVAILABLE
19

18. 20
Appsignal, Pingdom, Scalingo et Logmatic
MONITORING PERFORMANCE AND BUSINESS
20

19. 2121

20. 27-28 February 2017
LONDON
CONCLUSION

21. 23
•Test the value first
•Invest progressively on quality
•Create intimacy with the users
•Create a suitable environment for
experimentations
KEY TAKEAWAYS
23

22. Thank you!
27-28 February 2017
LONDON
https://hiptest.net