Talk/Workshop at the CI&T Digital Day 2015 in Belo Horizonte

1. A/B TESTING
In data we trust!
Pedro Marques CI&T Digital Day

2. PEDRO MARQUES
UX Designer @ booking.com
@pmarquees

3. 700k
Properties
135
Offices
207
Countries
42
Languages
74
Currencies
8.5k
Employees
8k
Servers
perl + html templates
Core
3º MAIOR
e-commerce do mundo

5. A/B Testing
WTF?

6. Emojiana

7. Emojiana Hippo

8. 50%
50%
11%
2%

9. Born at Google
Early 00’s

10. Pharmaceutical
To test new drugs

11. But why A/B Test?

12. No Regression

13. But bear in mind:

14. A/B Test will
prove you wrong

15. Data never lies

16. Define your KPI’s

17. Correlation does NOT
imply causation

18. Conver$ion

19. User performs
a given action

20. Define your KPI's

21. What should I test?

23. Test all your hypothesis

24. Assumption is the
mother of all fuck ups

25. Framing Effect

26. A/B Testing
Kills creativity

27. Junior designers X
Sr. Designes
https://medium.com/the-year-of-the-looking-glass/junior-designers-vs-senior-designers-fbe483d3b51e

30. Would like fries with that?
http://blog.booking.com/hamburger-menu.html

32. No significant impact.
At all.

33. NICE!
But how do I do that?

34. Coin flip

35. https://www.optimizely.com

36. Now what?
Data analysis is the key

37. Some advices
These are mine, test and see if it works for you =)

38. Shit X Shit
Nope, a big nope.

39. Ask questions
Even before starting the experiment

40. Dont trust the gurus
This is science, not religion

41. Flip the coin properly
Calculate the amount of time and sample that you need
https://vwo.com/ab-split-test-duration/

42. A/B Testing is a big subject

44. Link

46. Thank you folks! <3
hey@pedromarques.io
@pmarquees