This document discusses keys to success for optimization champions and creating great experimentation programs. It emphasizes that experimentation is important for driving motivation, insights and innovation. Great experimentation programs have five fundamentals - process, accountability, culture, expertise and technology. It provides perspectives from optimization champions at various companies on how they implement experimentation programs and priorities like establishing true north metrics and goals.

1. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome

2. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
You are the Catalyst: Optimization Champions’
Keys to Success
Chris Goward
Founder & CEO, WiderFunnel

3. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Conversion Rate Optimization

4. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Experimentation

5. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Experimentation
=/=
UX Improvements
Experimentation Drives:
Motivation
Insights
Business
Innovation
Delightful
Experiences

6. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
> 35,000
experiments per
year
> 30 Billion
variant combos
live at once
> 15,000
experiments per
year
7,000-10,000
experiments per
year

7. Tweet this: @ChrisGoward #Awesome
© 2007-2018 WiderFunnel Marketing Inc.
What creates great
experimentation programs?

8. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome

9. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome

10. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
1 2 3 4 5

11. Tweet this: @ChrisGoward #Awesome
© 2007-2018 WiderFunnel Marketing Inc.
The 5 Fundamentals for
Experimentation Programs

12. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Constraint

13. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Constraint

14. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Your Experimentation PACET™
Proces
s
Accountability
Culture
Expertis
e
Technology

15. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
“Most large organizations
embrace the idea of invention,
but are not willing to suffer the
string of failed experiments
necessary to get there.”
- Jeff Bezos

16. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Even in mature (Level 4)
organizations, optimizers say
experiments are
consistently derailed by
gut-feeling.
Source: “State of Experimentation Maturity 2018” research study by WiderFunnel, widerfunnel.com/research

17. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome

18. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Proven
Quantitative
Logical
Solid
Intuitive
Qualitative
Inspired
FuzzyYang
Yin

19. Tweet this: @ChrisGoward #Awesome
© 2007-2018 WiderFunnel Marketing Inc.
The experimentation mindset
should be embedded into your
process.

20. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
“If you can’t describe what you are
doing as a process, you don’t know
what you are doing.”
-W. Edwards Deming

21. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Your Experimentation PACET™
Proces
s
Accountability
Culture
Expertis
e
Technology

22. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Harper Grubbs
Director of Digital Marketing
www.Heifer.com
An Optimization
Champion’s
Perspective

23. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Harper Grubbs
Director of Digital Marketing
www.Heifer.com
An Optimization
Champion’s
Perspective

24. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Infinity Optimization Process

25. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Explore

26. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Explore
Your visitors convert in
their minds before they
convert with their click.

27. Tweet to: @chrisgoward

28. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Self-consistency Bias

29. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Self-consistency Bias
We like to keep consistent with what
we think, say and do, and will change
to ensure this is so.
- Cialdini, Influence.
(2006)

30. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Cognitive Dissonance
Cognitive Dissonance is mental stress
experienced when a person holds
contradictory beliefs, or acts in
contradiction to existing beliefs.
- Leon Festinger
(1957)

31. Tweet this: @ChrisGoward #Awesome
© 2007-2018 WiderFunnel Marketing Inc.
Is this the right idea to start with?
Let’s ask PIE

32. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Validate

33. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
PIE Prioritization Framework
Potential
How much improvement
can be made?
Ease
How easy would it be to
make these
improvements?
Importance
How much impact can
these improvements have
on our business goals?

34. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
PIE tells you where to experiment first
Lift Zone #1 8 7 7 7.3
Lift Zone Potential Importance Ease PIE Score
Web analytics
Heuristic & first impression analysis
Voice of customer
Revenue impact
Experimentation velocity
Cost of acquisition
Technical
“Political”

35. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
PIE tells you which ideas to prioritize
Hypothesis #1 3 7 9 6.3
Hypothesis Potential Importance Ease PIE Score
Digital Analytics evident
User Testing & Research evidence
Informed by Persuasion Principles
Drives a True North metric
Revalidates Past Insight
Informs Business Insight
Technical
“Political”

36. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome

37. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome

38. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Conversion Rate: +1.97%
Average donation value: +3.07%
Revenue per visitor: +5.10%

39. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
14% of visitors
68% of donors
86% of visitors
32% of donors
Selected
“Regular Donor”
Selected
“Occasional Donor”
Selected
“First Time Donor”
Non-Responders

40. Tweet this: @ChrisGoward #Awesome
© 2007-2018 WiderFunnel Marketing Inc.
Heifer can use this consumer insight
throughout all of their marketing.

41. Tweet this: @ChrisGoward #Awesome
© 2007-2018 WiderFunnel Marketing Inc.
And, we can optimize Personalized
data-informed experiences.

42. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Source: “State of Experimentation Maturity 2018” research study by WiderFunnel
Persuasion principles are overlooked by many
organizations
24% 80%
at the Level 1 Initiating
maturity level…
at the Level 4 Scaling
maturity level…
…are experimenting with persuasion principles

43. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Your Experimentation PACET™
Proces
s
Accountability
Culture
Expertis
e
Technology

44. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Nate Wallingsford
Head of US Marketing
Operations and Optimization
www.Fool.com
An Optimization
Champion’s
Perspective

45. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Nate Wallingsford
Head of US Marketing
Operations and Optimization
www.Fool.com
An Optimization
Champion’s
Perspective

46. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Business
Goals
Growth
Goals
Experiment
Goals
The Goals
Waterfall

47. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Johnny Russo
Associate Vice President,
Digital Marketing &
E-commerce
www.Marks.com
An Optimization
Champion’s
Perspective

48. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Johnny Russo
Associate Vice President,
Digital Marketing &
E-commerce
www.Marks.com
An Optimization
Champion’s
Perspective

49. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
True North Metric Examples
New users who make 10 friends in 7 days.
Number of nights booked per guest.
Teams who send 2,000+ messages.
Weekly Active Repinners.

50. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Your Experimentation PACET™
Proces
s
Accountability
Culture
Expertis
e
Technology

51. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Johnny Russo
Associate Vice President,
Digital Marketing &
E-commerce
www.Marks.com
An Optimization
Champion’s
Perspective

52. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Average # of team members dedicated to optimization
Initiating maturity level (Level 1 of 5)

53. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Average # of team members dedicated to optimization
Scaling maturity level (Level 4 of 5)

54. © 2007-2018 WiderFunnel Marketing Inc.
Tweet to: @ChrisGoward #Awesome
Your Experimentation PACET™
Proces
s
Accountability
Culture
Expertis
e
Technology

55. Discover how your experimentation program
stacks up
Get the new research report
Go to:
WiderFunnel.com/opticon2018

56. Q&A

57. Thank you!