We had the pleasure to present (Mobile) Web Testing and Optimizely at UX Torino's Meetup in Turin.

2. Lessons learned from
over 686,000 A/B testsMarco Sgrignuoli (Software Engineer)
Marco@optimizely.com
Pedro B. Meyer (Country Manager – Italy, Spain & Portugal)
Pedro@optimizely.com

3. Lesson #1:
Define Success

5. Lesson #1:
Define Quantifiable
Success Metrics

6. $ Per RecipientVisitors Per Day
Visit
Sign-Up Rate
Email $$

7. Media
Button

8. Button
● Sign Up
● Learn More
● Join Us Now
● Sign Up Now
Splash Page Experiment’s Variations
Media
• “Get Involved” Image
• “Family” Image
• “Change” Image
• Barack’s Video
• Springfield Video
• Sam’s Video

13. Button
● Sign Up
● Learn More
● Join Us Now
● Sign Up Now
Splash Page Experiment’s Variations
Media
• “Get Involved” Image
• “Family” Image
• “Change” Image
• Barack’s Video
• Springfield Video
• Sam’s Video

20. Button
● Sign Up
● Learn More
● Join Us Now
● Sign Up Now
Splash Page Experiment
Media
• “Get Involved” Image
• “Family” Image
• “Change” Image
• Barack’s Video
• Springfield Video
• Sam’s Video

22. Splash Page Experiment

23. Splash Page Experiment

24. Splash Page Experiment
Email
Subscriptions
Volunteers Amount Raised
Original 7,120,000 712,000 $143,000,000
+40.6% +2,880,000 +280,000 +$57,000,000
New 10,000,000 1,000,000 $200,000,000

25. Lesson #1:
Define Quantifiable
Success Metrics

26. Lesson #2:
Less Is More.
Reduce Choices.

31. Lesson #2:
Less Is More.
Reduce Choices.

32. Lesson #3:
Words Matter!
Focus on your Call to Action

33. Donation Button Experiment
Never
Signed-Up
Signed-Up
No Donation
Previously
Donated
0.0% 0.0% 0.0%
2.3% 27.8% 16.3%
-27.8 N/A N/A
15.2% -24.6% 11.9%
8.5% 2.9% 18.4%

35. Lesson #3:
Words Matter!
Focus on your Call to Action

36. Lesson #4:
Seek the Global Maximum

42. Lesson #4:
Seek the Global Maximum

43. Lesson #5:
Fail Fast

46. Lesson #5:
Fail Fast

47. Lesson #6:
Nothing is Sacred.
Challenge Assumptions.

50. Lesson #6:
Nothing is Sacred.
Challenge Assumptions.

51. Lesson #7:
Get Started Today!

57. 10,000+ Happy Customers!
alesport.com

58. Live Demo!