As an innovation center within HP, the Instant Ink team relies on an experimentation-led product strategy to challenge and validate business decisions. The experimentation program enriched with choice architecture and Behavioral Science reveals rich customers insights and accelerates their growth. Find out how you too can create an agile, digital-first, customer-obsessed program that drives product innovation. Since 2007, experimentation thought-leader Chris Goward has been building and scaling experimentation programs for both disruptors and traditional companies. In this session, he'll join Gretchen Gary, Senior Product Manager at HP, to share insights and real-world examples from fast-growth disruptors that any organization (including yours!) can profit from.

1. 1
 Sessions today will be recorded and will be available after
Opticon
 Join the conversation on Twitter at #Opticon19
 Like what you’ve seen at Opticon? Give feedback and rate
sessions on the mobile app

2. How Disruptors Experiment
Faster and Smarter
(and what you can learn from them)
@chrisgoward
Chris Goward Gretchen Gary
CEO, Widerfunnel Senior Product Manager, HP
@gretchengary

3. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
of organizations that
reported substantial year-
over-year revenue growth say
that decisions and actions
happen quickly.

4. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
2019 State of Experimentation Research Report

5. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
At your company,
how fast are decisions
and actions made
in everyday business?
Company size
An organization’s ability to move quickly is inversely correlated with
company size.

6. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
At your company,
how fast are decisions
and actions made
in everyday business?
Company size
An organization’s ability to move quickly is inversely correlated with
company size.
There’s still hope!

7. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19

8. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
The Innovator's Dilemma
Explore Exploit

9. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
The Innovator's Dilemma
Explore Exploit
OptimizationInnovation

10. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
Source: BCG Henderson Institute https://www.bcg.com/en-ca/publications/2018/2-percent-company.aspx

11. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
Characteristics of the 2% company
1. Excel at both exploration and exploitation.
2. Retain an “outside in” focus by bringing in outside
perspectives.
3. Embrace necessary disruptions, even when painful.
4. Have a clear model for renewal, bringing in outside innovation to
commercialize and optimize.
Source: BCG Henderson Institute https://www.bcg.com/en-ca/publications/2018/2-percent-company.aspx

12. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
The Innovator's Dilemma
Explore Exploit
How should we do
it best?
(Optimization)
What’s the next new
idea?
(Innovation)
Should we do
it?
(Validation)

13. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
The right experimentation process enables
testing along the spectrum.

14. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
Infinity Experimentation Process™
®

15. © Copyright 2007-2019 Widerfunnel. All rights reserved.
Explore
Expansive mindset,
looking for growth
opportunities with
data-informed
customer insights.

16. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
Instinct
Emotion
+ Reason
= Decision
The new understanding in brain research

17. Over 90% of
decisions are made
unconsciously, driven
by the brain’s limbic
system.
- Chris Goward

18. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
Cognitive Biases are important but have become overrated

19. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
Behavioral Science is much more than a
list of cognitive biases.

20. © Copyright 2007-2019 Widerfunnel. All rights reserved.
How well do you really know your customers?
• Age, gender, ethnicity, location?
• Where did the information come from?
• How often do you refine this?
• How have your customers changed over
time?
• What are their emotional states in their
buying scenarios?
Customer Persona
Sally Shopper
Customer Persona
Gerry Buyer

21. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
Source : The New Science of Customer Emotions, Harvard Business Review
Baseline
+13%
+52%
-18%
Not
Emotionally
Connected
Highly
Satisfied
Perceive
Brand
Differentiation
Fully
Connected
Emotional resonance of shopping experiences increases lifetime
value for highly satisfied customers

22. Your growth is
limited by your understanding of
your customers’
emotional needs.
- Chris Goward

23. To uncover insights about
customers’ underlying
emotional needs.
MotivationLabⓇ

24. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
LimbicⓇ emotional marketing framework
® Limbic is a registered trademark of Gruppe Nymphenburg Consult AG, used by Widerfunnel under exclusive North American license.

25. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
Emotional
State
Emotional
Type
Emotional
Context

26. © Copyright 2007-2019 Widerfunnel and HP Development Company, L.P.. All rights reserved.
@chrisgoward @gretchengary #Opticon19
Ink shoppers’ emotional type
Trust
Value
® Limbic is a registered trademark of Gruppe Nymphenburg Consult AG, used by Widerfunnel under exclusive North American license.

27. © Copyright 2007-2019 Widerfunnel and HP Development Company, L.P.. All rights reserved.
@chrisgoward @gretchengary #Opticon19
Ink shoppers’ emotional type + state
Trust
Value
Disciplined
® Limbic is a registered trademark of Gruppe Nymphenburg Consult AG, used by Widerfunnel under exclusive North American license.

28. © Copyright 2007-2019 Widerfunnel and HP Development Company, L.P.. All rights reserved.
@chrisgoward @gretchengary #Opticon19
Emotional Associations:
Creative, easy, artistic, crafty
Ink Quality:
High, but missing credibility
HP Instant Ink associations
The right ink, right on time, for less. Save up to 50% on ink. The most affordable way to print. Based on monthly subscription cost using only all pages in plan vs. cost per page of most color inkjet cartridge printers <
$399. Share of New Inkjet Unit Shipments (< $399) for 2018Q4 period in the US from IDC 2018Q4 Final Release. Standard cartridge CPP from gap intelligence 3/23/19. Never run out of ink. Based on plan usage,
Internet connection to eligible HP printer, valid credit/debit card, email address, and delivery service in your geographic area. Change or cancel at anytime. Change or cancel your plan at any time online except that
you can’t downgrade from a paid plan to the free plan since it is an introductory offer only. If you decide to cancel your HP Instant Ink plan you can go back to using HP original Standard or XL cartridges. Plan
upgrades are effective immediately and the charges will be applied retrospectively or in the next billing cycle, depending on user choice. Plan downgrades and cancellations are effective after the last day of the
current billing period. For full details go to instantink.hpconnected.com/terms.

29. © Copyright 2007-2019 Widerfunnel and HP Development Company, L.P.. All rights reserved.
@chrisgoward @gretchengary #Opticon19
Mis-aligned creative
Emotional
zone
Trust
Value
Discipline
® Limbic is a registered trademark of Gruppe Nymphenburg Consult AG, used by Widerfunnel under exclusive North American license.

30. © Copyright 2007-2019 Widerfunnel and HP Development Company, L.P.. All rights reserved.
@chrisgoward @gretchengary #Opticon19
Emotional Associations:
Value, Trust, Easy, Human
Ink Quality:
High, Credible
Save up to 50% on ink. The most affordable way to print. Based on monthly subscription cost using only all pages in plan vs. cost per page of most color inkjet cartridge printers < $399. Share of New Inkjet Unit Shipments (< $399) for 2018Q4 period in the US
from IDC 2018Q4 Final Release. Standard cartridge CPP from gap intelligence 3/23/19.
The right ink, right on time, for less. Save up to 50% on ink. The most affordable way to print. Based on monthly subscription cost using only all pages in plan vs. cost per page of most color inkjet cartridge printers <
$399. Share of New Inkjet Unit Shipments (< $399) for 2018Q4 period in the US from IDC 2018Q4 Final Release. Standard cartridge CPP from gap intelligence 3/23/19. Never run out of ink. Based on plan usage,
Internet connection to eligible HP printer, valid credit/debit card, email address, and delivery service in your geographic area. Change or cancel at anytime. Change or cancel your plan at any time online except that
you can’t downgrade from a paid plan to the free plan since it is an introductory offer only. If you decide to cancel your HP Instant Ink plan you can go back to using HP original Standard or XL cartridges. Plan
upgrades are effective immediately and the charges will be applied retrospectively or in the next billing cycle, depending on user choice. Plan downgrades and cancellations are effective after the last day of the
current billing period. For full details go to instantink.hpconnected.com/terms.
The new page

31. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
The evolution of HP Instant Ink

32. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
HP Instant Ink
• A unique “test” product home-grown within HP.
• Culture of experimentation, in store and online.
• Solving real customer problems with direct customer feedback.
• Tests inform data-driven decision making and projections for new
initiatives.
• Built trust in our program through process and protocol.
• Brought on Widerfunnel to advance the experimentation program.

33. © Copyright 2019 HP Development Company., LP All rights reserved.
@chrisgoward @gretchengary #Opticon19
A Timeline in Screens
20172015 2019
The right ink, right on time, for less. Save up to 50% on ink. The most affordable way to print. Based on monthly subscription cost using only all pages in plan vs. cost per page of most color inkjet cartridge printers <
$399. Share of New Inkjet Unit Shipments (< $399) for 2018Q4 period in the US from IDC 2018Q4 Final Release. Standard cartridge CPP from gap intelligence 3/23/19. Never run out of ink. Based on plan usage,
Internet connection to eligible HP printer, valid credit/debit card, email address, and delivery service in your geographic area. Change or cancel at anytime. Change or cancel your plan at any time online except that
you can’t downgrade from a paid plan to the free plan since it is an introductory offer only. If you decide to cancel your HP Instant Ink plan you can go back to using HP original Standard or XL cartridges. Plan
upgrades are effective immediately and the charges will be applied retrospectively or in the next billing cycle, depending on user choice. Plan downgrades and cancellations are effective after the last day of the
current billing period. For full details go to instantink.hpconnected.com/terms.

34. © Copyright 2019 HP Development Company., LP All rights reserved.
@chrisgoward @gretchengary #Opticon19
A Timeline in Screens
2016-20172015 2019
The right ink, right on time, for less. Save up to 50% on ink. The most affordable way to print. Based on monthly subscription cost using only all pages in plan vs. cost per page of most color inkjet cartridge printers <
$399. Share of New Inkjet Unit Shipments (< $399) for 2018Q4 period in the US from IDC 2018Q4 Final Release. Standard cartridge CPP from gap intelligence 3/23/19. Never run out of ink. Based on plan usage,
Internet connection to eligible HP printer, valid credit/debit card, email address, and delivery service in your geographic area. Change or cancel at anytime. Change or cancel your plan at any time online except that
you can’t downgrade from a paid plan to the free plan since it is an introductory offer only. If you decide to cancel your HP Instant Ink plan you can go back to using HP original Standard or XL cartridges. Plan
upgrades are effective immediately and the charges will be applied retrospectively or in the next billing cycle, depending on user choice. Plan downgrades and cancellations are effective after the last day of the
current billing period. For full details go to instantink.hpconnected.com/terms.

35. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #awesome
Choice Architecture moves beyond simple
optimization tactics.

36. © Copyright 2019 HP Development Company., LP All rights reserved.
@chrisgoward @gretchengary #Opticon19
Original choice: Should I activate Instant Ink?
Choice architecture – reframe the decision
The right ink, right on time, for less. Save up to 50% on ink. The most affordable way to print. Based on monthly subscription cost using only all pages in plan vs. cost per page of most color inkjet cartridge printers <
$399. Share of New Inkjet Unit Shipments (< $399) for 2018Q4 period in the US from IDC 2018Q4 Final Release. Standard cartridge CPP from gap intelligence 3/23/19. Never run out of ink. Based on plan usage,
Internet connection to eligible HP printer, valid credit/debit card, email address, and delivery service in your geographic area. Change or cancel at anytime. Change or cancel your plan at any time online except that
you can’t downgrade from a paid plan to the free plan since it is an introductory offer only. If you decide to cancel your HP Instant Ink plan you can go back to using HP original Standard or XL cartridges. Plan
upgrades are effective immediately and the charges will be applied retrospectively or in the next billing cycle, depending on user choice. Plan downgrades and cancellations are effective after the last day of the
current billing period. For full details go to instantink.hpconnected.com/terms.

37. © Copyright 2019 HP Development Company., LP All rights reserved.
@chrisgoward @gretchengary #Opticon19
New choice: Which Instant Ink plan should I choose?
Choice architecture – reframe the decision
+113.5%
The right ink, right on time, for less. Save up to 50% on ink. The most affordable way to print. Based on monthly subscription cost using only all pages in plan vs. cost per page of most color inkjet cartridge printers <
$399. Share of New Inkjet Unit Shipments (< $399) for 2018Q4 period in the US from IDC 2018Q4 Final Release. Standard cartridge CPP from gap intelligence 3/23/19. Never run out of ink. Based on plan usage,
Internet connection to eligible HP printer, valid credit/debit card, email address, and delivery service in your geographic area. Change or cancel at anytime. Change or cancel your plan at any time online except that
you can’t downgrade from a paid plan to the free plan since it is an introductory offer only. If you decide to cancel your HP Instant Ink plan you can go back to using HP original Standard or XL cartridges. Plan
upgrades are effective immediately and the charges will be applied retrospectively or in the next billing cycle, depending on user choice. Plan downgrades and cancellations are effective after the last day of the
current billing period. For full details go to instantink.hpconnected.com/terms.

38. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #awesome
Customer insights often yield more than
just improvements to conversion rate

39. © Copyright 2019 HP Development Company., LP All rights reserved.
@chrisgoward @gretchengary #Opticon19
New choice: What offer should I choose?
Choice architecture – reframe the decision, again!
+5.8%
The right ink, right on time, for less. Save up to 50% on ink. The most affordable way to print. Based on monthly subscription cost using only all pages in plan vs. cost per page of most color inkjet cartridge printers <
$399. Share of New Inkjet Unit Shipments (< $399) for 2018Q4 period in the US from IDC 2018Q4 Final Release. Standard cartridge CPP from gap intelligence 3/23/19. Never run out of ink. Based on plan usage,
Internet connection to eligible HP printer, valid credit/debit card, email address, and delivery service in your geographic area. Change or cancel at anytime. Change or cancel your plan at any time online except that
you can’t downgrade from a paid plan to the free plan since it is an introductory offer only. If you decide to cancel your HP Instant Ink plan you can go back to using HP original Standard or XL cartridges. Plan
upgrades are effective immediately and the charges will be applied retrospectively or in the next billing cycle, depending on user choice. Plan downgrades and cancellations are effective after the last day of the
current billing period. For full details go to instantink.hpconnected.com/terms.

40. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #awesome
Different customer groups respond differently:
• Cultural nuance across different countries
• Printer categories
• Retailer source
• Different points of the customer journey

41. © Copyright 2019 HP Development Company., LP All rights reserved.
Know in advance how best to evolve your solution. Explore, exploit, repeat.
Don’t Launch and Pray… Instead, Learn Then Launch

42. © Copyright 2019 HP Development Company., LP All rights reserved.
@chrisgoward @gretchengary #Opticon19
Var B: Colors + design
-40.6%
Control Var A: Rebrand
colors -2.1%
The brand design challenge
The right ink, right on time, for less. Save up to 50% on ink. The most affordable way to print. Based on monthly subscription cost using only all pages in plan vs. cost per page of most color inkjet cartridge printers <
$399. Share of New Inkjet Unit Shipments (< $399) for 2018Q4 period in the US from IDC 2018Q4 Final Release. Standard cartridge CPP from gap intelligence 3/23/19. Never run out of ink. Based on plan usage,
Internet connection to eligible HP printer, valid credit/debit card, email address, and delivery service in your geographic area. Change or cancel at anytime. Change or cancel your plan at any time online except that
you can’t downgrade from a paid plan to the free plan since it is an introductory offer only. If you decide to cancel your HP Instant Ink plan you can go back to using HP original Standard or XL cartridges. Plan
upgrades are effective immediately and the charges will be applied retrospectively or in the next billing cycle, depending on user choice. Plan downgrades and cancellations are effective after the last day of the
current billing period. For full details go to instantink.hpconnected.com/terms.

43. © Copyright 2019 HP Development Company., LP All rights reserved.
@chrisgoward @gretchengary #Opticon19
Control: Widerfunnel’s Var A: Internal design team -14.6%
The right ink, right on time, for less. Save up to 50% on ink. The most affordable way to print. Based on monthly subscription cost using only all pages in plan vs. cost per page of most color inkjet cartridge printers <
$399. Share of New Inkjet Unit Shipments (< $399) for 2018Q4 period in the US from IDC 2018Q4 Final Release. Standard cartridge CPP from gap intelligence 3/23/19. Never run out of ink. Based on plan usage,
Internet connection to eligible HP printer, valid credit/debit card, email address, and delivery service in your geographic area. Change or cancel at anytime. Change or cancel your plan at any time online except that
you can’t downgrade from a paid plan to the free plan since it is an introductory offer only. If you decide to cancel your HP Instant Ink plan you can go back to using HP original Standard or XL cartridges. Plan
upgrades are effective immediately and the charges will be applied retrospectively or in the next billing cycle, depending on user choice. Plan downgrades and cancellations are effective after the last day of the
current billing period. For full details go to instantink.hpconnected.com/terms.

44. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #awesome
What worked for Instant Ink?
• Created as a separate business unit.
• Direct voice of customer.
• Results-based pilot created a culture of experimentation.
• Solving for real customer problems.
• Outsider’s perspective with partners like Widerfunnel.

45. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
Consider customer satisfaction in your
Overall Evaluation Criterion (OEC).
Don’t just optimize for conversion rate.
A/B tested insight.

46. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
Design for conversion is a unique skill,
best suited for direct response
touchpoints.
A/B tested insight.

47. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
If the choice is complicated,
try reframing it.
A/B tested insight.

48. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
Instead of multiple options, try
delaying the more complicated decision.
A/B tested insight.

49. © Copyright 2007-2019 Widerfunnel. All rights reserved.
@chrisgoward @gretchengary #Opticon19
Get the latest insights on the industry
with the 39-page research report

50. Free Download for
Opticon attendees
Email:
iwant@Widerfunnel.com
Or visit:
Widerfunnel.com/Opticon19

51. Architecting insights.
Delivering growth. ™
Questions?

52. Thank you!

53. Session Exit Slide
 Refreshments now being served in the Expo Hall
 Fireside Chat with Optimizely Co-founder Dan Siroker at
2:15pm
 Closing Keynote with Dr. Mae Jemison at 2:45pm