5. #webclinic
5
Marketing takes time and resources
The Ideal CustomerThe Marketer
The
Message
No
Thanks
Time
Energy
Budget
? How can I know that my campaigns will be
successful before I invest resources?
6. #webclinic
6
Background: Medical provider specializing in treating a particular chronic
pain.
Goal: To plan a content marketing strategy based on which approach
generates more appeal in condition-based searchers.
Primary Research Question: Which content approach will achieve a higher
clickthrough rate?
Approach: A/B Multifactor Split Test
Experiment ID: Research Partner Identity Protected
Location: MarketingExperiments Research Library
Test Protocol Number: TP4067
Research Notes:
Experiment: Background
7. #webclinic
Experiment: Medical Condition Keywords
Medical Condition Keywords:
• Keyword 1
• Keyword 2
• Keyword 3
• Keyword 4
• Keyword 5
• Keyword 6
Ad Abbreviated Med. Conditions:
• Thyroid pain
• Sciatic nerve
• Blood Pres.
• Kidney cysts
• Colonic pain
• Lumbar pain
*Keywords have been randomly selected and optimized to protect partner identity
8. #webclinic
Experiment: Treatments
Treatment Design:
• Template test in 6 condition-
based ad groups
• Headline is specific to
[condition] ad group
• Variable is description line one
• Value corresponds to content
approach
[Condition] Sufferer?
Free access to kidney pain resources
from the experts in kidney health.
[Display URL]/[condition]
T1: Body Part Pain Resources
9. #webclinic
Experiment: Treatments
[Condition] Sufferer?
Compare available treatments,
from the experts in [specialty] health.
[Display URL]/[condition]/[condition]
T2: Treatment Options
Treatment Design:
• Template test in 6 condition-
based ad groups
• Headline is specific to
[condition] ad group
• Variable is description line one
• Value corresponds to content
approach
10. #webclinic
Experiment: Treatments
[Condition] Sufferer?
Learn about the causes & solutions,
from the experts in [specialty] health.
[Display URL]/[condition]
T3: Causes and Solutions*
Treatment Design:
• Template test in 6 condition-
based ad groups
• Headline is specific to
[condition] ad group
• Variable is description line one
• Value corresponds to content
approach
*Audience member submission
11. #webclinic
Experiment: Treatments
[Condition] Sufferer?
How to recognize the symptoms,
from the experts in [specialty] health.
[Display URL]/[condition]
T4: Recognize Symptoms
Treatment Design:
• Template test in 6 condition-
based ad groups
• Headline is specific to
[condition] ad group
• Variable is description line one
• Value corresponds to content
approach
12. #webclinic
Experiment: Side-by-side
[Condition] Sufferer?
Free access to specific pain resources
from the experts in [specialty] health.
Display URL/[condition]
T1: specialty Pain
Resources [Condition] Sufferer?
Compare available treatments,
from the experts in [specialty] health.
Display URL/[condition]
T2: Treatment Options
[Condition] Sufferer?
Learn about the causes & solutions,
from the experts in [specialty] health.
Display URL/[condition]
T3: Causes and Solutions*
[Condition] Sufferer?
How to recognize the symptoms,
from the experts in [specialty] health.
Display URL/[condition]
T4: Recognize Symptoms
*Amy’s submission
13. #webclinic
Version CTR Rel. diff Stat. Conf
[Specialty] Pain Resources 0.15% - -
Treatment Options 0.17% 14.1%
Causes & Solutions* 0.26% 71.8%
Symptoms 0.30% 99.7%
99.7% Increase in clickthrough rate
‘Symptoms’ generated 99.7% more clicks than the ‘[Specialty] Pain
Resources’
What You Need to Understand: The ‘Symptoms’ content approach is most
appealing to visitors making condition-based searches.
*Audience submission
Experiment: Results
99.9%
99.9%
65.8%
14. #webclinic
14
The Ideal CustomerThe Marketer
The
Message
No
Thanks
Time
Energy
Budget
? How can I know that my campaigns will be
successful before I invest resources?
How can we predict success?
15. #webclinic
Today we are going to learn 3 principles to help us
predict the success of our campaigns.
17. #webclinic
Key Principles
PRINCIPLE #1:
Customer research can generate an overwhelming amount of data. We can
cut through it with this simple maxim:
All we need to know is enough to predict customer behavior.
How can we predict success?
18. #webclinic
Knowing enough to predict
Our researchers call this knowledge the “Customer Theory.”
Customer Theory Definition:
The Customer Theory is an understanding of the customer
that enables us to predict their total response to a given offer.
19. #webclinic
Knowing enough to predict
[Condition] Sufferer?
Free access to [specialty] pain
resources
from the experts in [specialty] health.
[Display URL]/[condition]
T1: [Specialty] Pain Resources
[Condition] Sufferer?
Compare available treatments,
from the experts in [specialty] health.
[Display URL]/[condition]
T2: Treatment Options
[Condition] Sufferer?
Learn about the causes & solutions,
from the experts in [specialty] health.
[Display URL]/[condition]
T3: Causes and Solutions*
[Condition] Sufferer?
How to recognize the symptoms,
from the experts in [specialty] health.
[Display URL]/[condition]
T4: Recognize Symptoms
*Amy’s submission
? What can we predict based on these results?
20. #webclinic
Experiment #2: Background
Background: Medical provider specializing in treating chronic pain.
Goal: To plan a content marketing strategy based on which approach
generates more appeal in condition-based searchers.
Primary Research Question: Which content approach will achieve a higher
clickthrough rate?
Approach: A/B Multifactor Split Test
Experiment ID: Research Partner Identity Protected
Location: MarketingExperiments Research Library
Test Protocol Number: TP4068
Research Notes:
22. #webclinic
General Hypothesis:
“Based on what we learned
from the previous content
approach test, if we use a
symptom content approach
while matching the control's
specificity to each ad group,
we can achieve a higher click-
through rate.”
-Quoted from Test Protocol
Experiment #2: Control Keyword 1 Treatment
Read about our xx-minute treatment
for keyword pain relief.
[Display URL]/Keyword_1
Keyword 2 Treatment
Keyword pain? You Have Options.
Ask Our Experts About [procedure].
[Display URL]
Keyword 3 Treatment
Relieve [keyword] Pain with
Our Unrivaled xx-Minute Procedure.
[Display URL]/Keyword 3
Degenerative Keyword Disease
Relieve Degenerative [keyword] Pain w/
Our Unrivaled Revolutionary Program
[Display URL]/Degenerative
Keyword 4 Treatment
New 53-Minute Procedure, Reclaim
Your Life From [keyword] Pain.
[Display URL]/Keyword
Relief From Keyword Pain
Read about our 30-minute treatment
for total freedom from [keyword] pain!
[Display URL]
Control Ads
23. #webclinic
Experiment #2: Treatment 1
Keyword 1 Symptoms
Read about our XX-minute treatment
For keyword pain relief.
[Display URL]/keyword
Keyword 2 Symptoms
Keyword 2? You Have Options.
Ask Our Experts About [Procedure™].
[Display URL]
Keyword 3 Symptoms
Relieve [keyword] Pain with
Our Unrivaled XX-Minute Procedure.
[Display URL]/keyword
Keyword 4 Symptoms
New xx-Minute Procedure, Reclaim
Your Life From [keyword] Pain.
[Display URL]/keyword
Keyword 5 Symtpoms
Read about our XX-minute treatment
for total freedom from [keyword] pain!
[Display URL]
Treatment 1 Ads
How treatment will test
hypothesis:
“If treatment 1 wins, we will
learn that the symptom
content approach is most
effective only when used in
the headline.”
-Quoted from Test Protocol
24. #webclinic
How treatment will test
hypothesis:
“If treatment 2 wins, we will
learn that the symptom
content approach is most
effective when used in the
description and when the
description is specific to the
ad group.”
-Quoted from Test Protocol
Experiment #2: Treatment 2
Keyword 1 Treatment
Read about our xx-minute treatment
for keyword pain relief.
[Display URL]/keyword
Keyword 2 Treatment
Read about keyword Symptoms
and see how our experts can help!
[Display URL]
Keyword 3 Treatment
Read about keyword Symptoms
and see how our experts can help!
[Display URL]/keyword
Degenerative Keyword Disease
Read how our keyword experts can help
relieve Degenerative keyword
Symptoms!
[Display URL]/Degenerative
Keyword 4 Treatment
Read about keyword Symptoms
and see how our experts can help!
[Display URL]/keyword
Relief From Keyword Pain
Read about keyword Symptoms
and see how our experts can help!
[Display URL]
Treatment 2 Ads
25. #webclinic
Experiment #2: Treatment 3
How treatment will test
hypothesis:
“If treatment 3 wins, we will
learn that the symptom
content approach is most
effective when used in BOTH
the headline and description
and when the description is
specific to the ad group.”
-Quoted from Test Protocol
Keyword 1 Symptoms
Read about our xx-minute treatment
for keyword pain relief.
[Display URL]/keyword
Keyword 2 Symptoms
Read about keyword Symptoms
and see how our experts can help!
[Display URL]
Keyword 3 Symptoms
Read about keyword Symptoms
and see how our experts can help!
[Display URL]/ /keyword
Keyword 3 Symptoms
Read about keyword Symptoms
and see how our experts can help!
[Display URL]/ /keyword
Keyword 4 Symptoms
Read about keyword Symptoms
and see how our experts can help!
[Display URL]
Treatment 3 Ads
26. #webclinic
Experiment #2: Results
Version CTR Rel. diff Stat. Conf
Control 0.28% - -
T1- Headline Change 0.26% -5.8%
T2- Description Change 0.21% -26.5%
T3- Both Change 0.39% 40.6%
40% Increase in clickthrough rate
Adding ‘Symptoms’ to BOTH headline and description produced a 40% increase
What You Need to Understand: By applying the insight from the previous
test and inserting ‘symptoms’ into both the headline and description, the
team was able to create more successful treatments across all ad groups.
99.9%
99.9%
60.6%
27. #webclinic
27
Knowing enough to predict
Which? Why? What?
Research Question Theory Question
Value Proposition Question:
If I am your ideal customer, why should I buy from you
rather than your competitors?
Ideal Customer
Behavior reflects identity
28. #webclinic
28
201% 2% 29%
Test results are interpreted and
second test was created based on
the analyst’s observations
Again, test results are
interpreted and the next
round of testing is started
for this page
Test is again interpreted and
transferrable principles are
applied to other offer pages
Knowing enough to predict
Customer Theory applied
29. #webclinic
29
451%
302%
603%257% 28%
• The discoveries and insights
about customer motivation
from the three prior tests were
applied to other landing pages
and used to optimize PPC
campaigns.
• The purposeful effort to
identify and selectively apply
these transferrable insights led
to widespread optimization
gains.
Knowing enough to predict
Customer Theory applied
31. #webclinic
Key Principles
PRINCIPLE #1:
Customer research can generate an overwhelming amount of data. All we need
to know is enough to predict customer behavior.
PRINCIPLE #2:
The Internet is not just another marketing channel, it is the primary laboratory
for conducting customer research.
How can we predict success?
32. #webclinic
32
The Internet as a lab
The Internet has become the most efficient
means of gathering business intelligence
BEFORE a major online (or offline) campaign.
Level 1
Level 2
Level 3
The Decision
Resolution
“To know what
people really think,
pay regard to what
they do, rather than
what they say.”
– Descartes (source
unknown)
33. #webclinic
33
The Internet as a lab
Traditional marketing process
The Traditional Marketing Process
Business
Leader Creates
Product
Agency delivers
creative
Marketing
Implements
Online marketing has simply been
ADDED to the process.
34. #webclinic
34
The Behavior-Based Marketing Process
Business
Leader Creates
Product
Agency delivers
creative
Marketing
Implements
If we can interrupt the process and
test the message, then we can drive
results across the business.
The Internet as a lab
Behavioral-based marketing process
35. #webclinic
35
The Internet as a lab
548%
SEO Landing Page
Example
141%
Homepage
Lower investment
channels
Higher
investment
channels
Offline Campaigns
*TV image by Brylle
37. #webclinic
Key Principles
PRINCIPLE #1:
Customer research can generate an overwhelming amount of data. All we need
to know is enough to predict customer behavior.
PRINCIPLE #2:
The Internet is not just another marketing channel, it is the primary laboratory
for conducting customer research.
PRINCIPLE #3:
The goal of a test is not (simply) to get a lift but rather to get a learning. Even a
negative test can translate into a positive result.
How can we predict success?
38. #webclinic
38
Background: Company is a publisher of electronic marketing information and offers
related services.
Goal: Increase registrations for a free email newsletter.
Primary Research Question: Which sign-up page will yield the highest conversion rate?
Approach: A/B/C Multivariate test involving changes in headline, credibility indicators,
and images according to optimization best practices.
Experiment ID: (Protected)
Location: MarketingExperiments Research Library
Test Protocol Number: #TP1092
Research Notes:
Experiment #3: Background
41. #webclinic
41
• Clearer headline emphasizes
the value proposition.
• “Featured Clients” list
emphasizes value and reduces
anxiety.
• Bolded key terms make body
copy easier to read and scan.
• Form fields are reduced to the
bare minimum.
• Costumer testimonials reduce
anxiety.
• Anti-spam seal reduces
anxiety.
Treatment 1
Experiment #3: Treatment 1
42. #webclinic
42
• Headline is stronger and
more quantitative to
emphasize the value
proposition.
• Added more testimonials.
• Rearranged page to give
better site flow.
• Added personal feel with
images and hand-written
signature.
• “Tell me where to send…”
language used.
Treatment 2
Experiment #3: Treatment 2
46. #webclinic
46
Version CR Rel. diff Stat. Conf
Original 14.26% - -
Treatment 1 6.74% -52.7%
Treatment 2 6.84% -52.0%
53% Decrease in conversion
The lowest performing treatment decreased conversion by 53%
What You Need to Understand: Through the negative lift in this test, we
were able to learn that the customer was already sold by the time they
landed on this page.
95%
95%
Experiment #3: Results
47. #webclinic
47
• Common landing page best
practices failed to improve
conversion on this original
page.
• Began testing removal of
elements from the page to
match visitor motivation
levels.
• If adding elements to
increase the value
proposition decreased
conversion, maybe the traffic
to this page was already
highly motivated?
Control
Experiment #4: Control
48. #webclinic
48
• Much of the copy on
this page is removed,
leaving simple form
submission fields.
• No real selling points
are included in this
design.
Treatment
Experiment #4: Treatment
50. #webclinic
50
Version CR Rel. diff Stat. Conf
Original 12.09% - -
Treatment 21.54% 78.12%
78% Increase in conversion
The treatment increased conversion by 78.12%
95%
Experiment #4: Results
51. #webclinic
51
What You Need to Understand: Without the “negative lifts” that
challenged almost every part of our normal optimization methodology, it
would have taken much longer to achieve the eventual 78% increase.
X X
Test Cycle: Results
53%
52% 78%
52. #webclinic
52
What You Need to Understand: Without the “negative lifts” that
challenged almost every part of our normal optimization methodology, it
would have taken much longer to achieve the eventual 78% increase.
Test Cycle: Results
53%
52% 78%
53. #webclinic
53
Summary: Putting it all together
Key Principles
PRINCIPLE #1:
Customer research can generate an overwhelming amount of data. All we need
to know is enough to predict customer behavior.
PRINCIPLE #2:
The Internet is not just another marketing channel, it is the primary laboratory
for conducting customer research.
PRINCIPLE #3:
The goal of a test is not (simply) to get a lift, but rather to get a learning. Even a
negative lift can translate into a positive result.
63. #webclinic
63
Submission 3
Primary Channel:
Organic keywords- home lighting;
ceiling lights; table lamps;
We just updated the page and lost
ranking.
Target Audience:
Home owners 40+ and contractors.
Page Objective:
The next action should be to search,
and once they search they should
buy.
We plan to have a sign-up box on the
search page so users can enter their
email to receive special offers.
http://kewlighting.co.uk/
65. #webclinic
65
Submission 5
Primary Channel:
Display advertising
(Sympatico, Facebook,
Olive Network, Casale,
Cineplex) and paid search
Target Audience:
Future students (main
target grade 12 grads -18-
24, but extending as wide
as to 35; mainly in Ontario
and western Quebec)
Page Objective:
To click “book a tour” or
“request a publication”
http://www.algonquincollege.com/prospective/
66. #webclinic
MECLABS conducts rigorous experiments in the new science of optimization.
We apply our discoveries to help leaders optimize the financial performance
of their sales and marketing programs.
Learn more about how you may
be a fit for a MECLABS research
partnership:
• Select Research Partnership
Opportunities on the post-
webinar survey
• Contact us directly
info@MECLABS.com
1-877-635-0565
MECLABS Research Partnership Opportunities
x