The document provides an overview of the state of the email marketing industry and best practices. It discusses that: 1) Industry performance is strong, with email bringing in $40.56 for every dollar spent. Open and click-through rates vary by industry but are around 20% and 5% on average. 2) Capturing opt-ins everywhere and using techniques like popovers, which can increase conversions by 200-400% despite being annoying, is key to growth. 3) The shift is from broadcast messaging to more personalized, automated communications based on behaviors and preferences. Relevant messages see a 9x increase in sales. In 3 sentences or less, I have summarized the key points made

1. 9/18/2012
State of the Email Marketing
Section Title
Industry
Speaker
• Loren McDonald
– VP, Industry Relations
– Twitter: @LorenMcDonald
– Google+: +LorenMcDonald
1

2. 9/18/2012
Email is
Strong
Opt‐in
Q & A
Everywhere
Agenda
Design for
Automation
Mobile
Inactives Content
1. Industry performance is strong
- email still rocks
2

3. 9/18/2012
Email is thriving by ALL accounts!
Email is bringing in $40.56 for
every dollar spent on it.
Source: DMA
3

4. 9/18/2012
User Base Still Growing
4

5. 9/18/2012
20.1%
Open Rate ‐ Average
43.7%
Open Rate ‐ Top Quartile
8.0%
Open Rate ‐ Bottom Quartile
Open Rate – Select Industries (WW)
Top
Vertical Mean
Quartile
Retail 17.12% 30.60%
Leisure, Sport &
16.18% 32.61%
Recreation
Media 20.91% 47.98%
Banks / Fin. Services 22.62% 48.91%
5

6. 9/18/2012
5.2%
Click‐through Rate‐ Average
16.6%
WW: CTR ‐ Top Quartile
0.7%
CTR ‐ Bottom Quartile
CTR – Select Industries (WW)
Top
Vertical Mean
Quartile
Retail 3.09% 6.21%
Leisure, Sport &
2.31% 5.29%
Recreation
Media 8.87% 31.67%
Banks / Fin. Services 3.46% 9.88%
6

7. 9/18/2012
19.3%
Click‐to Open Rate‐ Average
40.0%
CTOR ‐ Top Quartile
6.7%
CTR ‐ Bottom Quartile
2. Capturing opt-ins and data
everywhere your customers are
7

8. 9/18/2012
Popovers: A Bit Annoying –
But They Work!!!
Pop‐overs = 200‐400% lift
38% of Americans
would rather
clean their toilet than
create another
Username/password
combination!
16
8

9. 9/18/2012
What types of data are available?
9

10. 9/18/2012
Facebook Timeline Email Sign Up
Tablet/Kiosk Opt‐in Collection
10

11. 9/18/2012
SMS to Email Opt‐in
• POS
• Garden Center Tips
• Video Alerts
• Return/Info. center
• Store Signage
• Events
QR Codes to Email Opt‐in
• POS
• Garden Center Tips
• Video Alerts
• Return/Info. center
• Store Signage
• Events
11

12. 9/18/2012
State of QR Code Usage/Awareness
– Still A Little Early
3. The shift from broadcast to
automation + behavior
12

13. 9/18/2012
Pounding isn’t the answer, because…
…Hope is not an marketing strategy.
13

14. 9/18/2012
Individual relevance is
increasingly the key to
deliverability
Broadcast to Behavior
Marketing Sophistication
Bto1
Individual
Messages
Behavior‐based
Messages
(Multi‐Track)
Time‐based Messages
Automated (Drip, Simple Nurture)
Manual
Targeted Mailings based on Segmentation
Mass Mailing to a broad database
Response Rates/
Engagement
14

15. 9/18/2012
Relevant Emails Rock!
9x
Note: Metrics and salary costs based on JupiterResearch executive survey. Broadcast assumes
$3 CPM, and all others assume $4.5 CPM. All assume $89 AOV, 50 percent product margin,
and 2.8 million pieces of mail per month.
Low Volume, High ROI!
Volume of Emails sent Sales Generated
4.1%
Triggered
Campaigns
40.2%
Triggered
Campaigns 59.8 %
95.9 % Batch
Batch Campaigns
Campaigns
15

16. 9/18/2012
Explicit data
Demo‐
graphics
Automation
Behavior
Implicit data
Programs – Mass Personal
16

17. 9/18/2012
Browse Behavior
Browse Behavior
Customer viewed “Joint Supplements” landing page Sent SmartFlex
Finder email
17

18. 9/18/2012
Message A – 1 day after
Message B – 3 days after
18

19. 9/18/2012
Message C – 5 days after
DEMCO Cart Results
Cart Email A Cart Email B Cart Email C
Day 1 Day 3 Day 5 AVERAGE
Open Rate 40% 39% 32% 37%
Click-to-Open 44% 47% 28% 41%
Click-thru-Rate 18% 18% 9% 15%
Conversion Rate 22% 15% 24% 20%
Sales/email $8.60 $8.40 $5.04 $7.46
19

20. 9/18/2012
4. Content and personality plays
a bigger role
[get pur‐suh‐nl]
1. Marketing content that speaks
with a “human” voice.
2. Targeted, personally relevant
messaging based on consumer
preferences, demographics and
behavior.
3. Dude, lose the corporate speak.
20

21. 9/18/2012
“Human” Cart
abandonment
• Aaron -
Real person
• Service
tone
• 50%
conversion
rate
21

22. 9/18/2012
Let Customers Do the Selling
30+% Increase in Orders and Sales
22

23. 9/18/2012
How‐to video
Important Dates Reminder Email
46
23

24. 9/18/2012
Your dates trigger an email…
21 days before occasion
Personalised with name
Occasion w/ date
Coupon code
10% Off
31% CTR
Highest revenue
generating email
5. Inactives become the biggest
challenge
24

25. 9/18/2012
Inactives Are a HUGE Issue
Typical Actives vs. Inactives Ratio
25

26. 9/18/2012
Impact of Inactives
Lost potential revenue
Inefficiencies/Reduced ROI
Fuzzy metrics
Potential reduced deliverability
Hypothetical Revenue Loss
Actives Inactives
# Subscribers 250,000 200,000
Average Revenue Per
$0.12
Email Delivered
4 X week X 53 weeks
$6.4M
~ 212 emails
Potential Lost
Revenue $5.1M
26

27. 9/18/2012
Reactivation & List Cleansing
• Data cleansing by implementing a Re Engagement Program
• Contacts who have had no mailing activity for 9 months
• Have not opted in within the last 3 months
Re‐engagement Email 1
Birmingham Airport: Hello
There!
27

28. 9/18/2012
3 weeks later…Re‐engagement Email 2
Birmingham Airport: We Miss You!
2 more weeks later…Final Email
Birmingham Airport: Details deleted
28

29. 9/18/2012
Data Cleansing
Upon receiving Email 3 contacts are
track routed to ‘To Be Removed
from Database’ track and then get
purged by the client
Upon receiving Email 3 contacts’
profiles get date stamped under
‘Inactive’ date database field which
makes them easy to query off
The best reactivation strategy
is to minimize inactives
to begin with.
29

30. 9/18/2012
Typical Reactivation Rate: 1%‐2%
High Performers: 10%+
Automate: Activate Early Inactives
New subscribers don’t
open/click first XX messages/X
weeks
Move these early inactives into
“activation” track
Send survey, different offers;
best of, diff subject lines,
testimonials, etc.
30

31. 9/18/2012
Opt‐in process
Preference mgmt. Expectation mgmt.
Minimize!
Targeted / Relevant Welcome /
Content Onboarding
6. Designing for mobile is no
longer optional
31

32. 9/18/2012
Going Mobile / Screensize‐apalooza
Smartphone Sales Pass PC Sales
Q4 2010
32

33. 9/18/2012
U.S. Smartphone Penetration –
Nielsen 2/2012
Tablets…
369M
2016
Source: Gartner, April 2012
33

34. 9/18/2012
35% of email subscribers are
opening your emails on their
mobile device!
Where is Your Email Being Read?
34

35. 9/18/2012
So Context Means…
Scannable
Singular calls to action
Larger fonts
Single columns
Bullet‐proof / large CTA buttons
Design for touch …
The New Design Challenge
Old mouse New mouse
35

36. 9/18/2012
Do Your Email’s Have the Touch?
Source: StyleCampaign.com, Litmus
Know Your Audience
Source: Unica Corporation/Pivotal Veracity
36

37. 9/18/2012
Regular
Mobile
64% higher
CTR
A/B test sent
to frequent
mobile
openers
Thanks / Q & A
Loren McDonald
lmcdonald@silverpop.com
@LorenMcDonald
Google+: LorenMcDonald
37

38. 9/18/2012
Anatomy of an Email
Marketing Program
Agenda
• Introductions
• Rules
• State of the Email Industry & Top
Challenges
• Foundational Program Overview
– Acquisition
– Welcome
– Promotion
– Transactional
– Opt-Out
• Conclusion
1

39. 9/18/2012
Goal for Today
Teach you how to optimize
your foundational programs
so that you can be effective
at advanced strategies
You need to be here
100k
50k
25k
You are here
Ground Level
2

40. 9/18/2012
Hello, my name is Ryan
• Industry Experience
– Director, Email Marketing & Acquisition at
Sears Holdings
– Responsible for East Coast Operations at
Responsys
– Director, Email Marketing & Online
Advertising at infoGROUP, Inc
• Thought Leadership
– DM News : Email Gets Personal (Cover
Story)
– Keynote address – March 2012, EEC12
– Recent Article: Changing Consumer
Perceptions of the Email Channel, Simms
Jenkins - bit.ly/RP_P1
Ryan Phelan – Co-Chair of the EEC List Growth &
Engagement Roundtable
Vice President, Strategic Services
– Member of:
Various Email Discussion Groups on:
Hello, my name is Damian
What I do
• Oversee Strategy, Creative,
Deliverability, Integration, Accounts
and Campaign Management
• Focus on retention marketing
• If I do my job, then we have a lot of
happy clients with great email
programs
Thought Leadership
• Speaker at Email Insider Summit
Damian Borichevsky 2011 and 2012, Email Evolution
SVP, Professional Services 2012, Casual Connect 2012
PulsePoint
3

41. 9/18/2012
Rules
• Opinions are welcome
• This is a discussion, not
presentation
• You know your business,
share it
• All opinions, debates,
questions, leaps of faith,
gut feelings are valid
A Look at the Marketplace
What’s really happening out there
4

42. 9/18/2012
Dispel the myth
Why do marketers
believe that every
message we send evokes
this reaction?
Marketing Rationalization
5

43. 9/18/2012
Challenges for Email Marketers
• Deliverability
– Challenges with Yahoo!
across the industry
– Changes in how “Spam”
is defined
• Relevancy and the a
relationship to the
consumer
• Time, Resources, Money
and Smart Marketing
STOP THE INSANITY!
6

44. 9/18/2012
1,000 consumers were surveyed…
• U.S. Based Consumers
• 77% own a smartphone
• Selected ages between 18 and 45
• 79% were employed
• Diverse ethnic mix
• 77% earned $35,000+
• 35% live in urban area; 65% in
suburb
How many email addresses do
you have?
© Digital River Inc. , Source: BlueHornet Online Email Behavior Survey, Feb 2012 n=1,000
7

45. 9/18/2012
How many times a day do you
check your personal email?
© Digital River Inc. , Source: BlueHornet Online Email Behavior Survey, Feb 2012 n=1,000
Why do you sign up for email?
© Digital River Inc. , Source: BlueHornet Online Email Behavior Survey, Feb 2012 n=1,000
8

46. 9/18/2012
Change must happen
Consumers
will
continue
to grow
weary of
the email
onslaught
What is Engagement Marketing?
9

47. 9/18/2012
Engagement Marketing
The combination of customer disclosed preference with
behavioral based data to form an actionable picture of
the customer that allows the marketer to answer the
needs of the consumer through active communication.
Simply put, engagement
marketing is selling
without looking like you’re
selling
Giftcertificates.com
Primary message
is thankful, not
BUY, BUY, BUY!
10

48. 9/18/2012
Foundational Email Programs
Getting on the path to success
Email Program Development
Strategic Planning: Advanced Persona Development, Cluster Analysis, Next Logical Product, Behavioral Segmentation,
Attitudinal, Dynamic Messaging, Shopping cart abandonment, Transactional Messaging,
programs that rely on big data
Video in Email, Preference Center Phase 2, Segmentation, Differentiated Welcome
and progressive thinking. Messaging, Social Messaging
Revenue Impact: Extreme
Foundational Programs: Those
programs that are common and Welcome, Transactional, Opt-Down, Basic Acquisition, Basic Segmentation, Promotional,
expected from the consumer Cadence Guidelines, List Usage Rules, Testing
Revenue Impact: Substantial
Low Hanging Fruit: Those quick
fixes that can correct a programs Calls To Action, Cadence, Design, Capture, Opt-In, Shopping Cart, Product Placement,
bad path. Rules and Legislation
Revenue Impact: Minimal
11

49. 9/18/2012
Agenda
• Foundational Program
Optimization
– Acquisition
– Welcome
– Promotion
– Transactional
– Opt-Out
Acquisition
Providing the food for your program
12

50. 9/18/2012
Acquisition Defined
Generically…
• Acquisition (a-kwə-ˈzi-shən): coming into possession or
control of often by unspecified means (Merriam-Webster)
In the context of Email…
• Subscriber Acquisition (səb-ˈskrīb-ər a-kwə-ˈzi-shən):
Convincing people to provide both their email address
and their consent for you to distribute content to that
address. (Borichevsky-Phelan)
Where Should You Look?
• Who are you trying to acquire?
• What are you willing to pay for a subscriber?
• How fast are you trying to grow?
• Where is your quality vs. quantity slider?
13

51. 9/18/2012
Channel Options
• Email
– Effective…if you do it right
• Display
– Make sure you are contextually targeting your ad
• Social
– Very cost effective, but be wary of adding a bunch of
people who just want to win something
• Co-Registration
– Make sure you find a white hat provider
• Appending
– Not a best practice – You are assuming consent
What should you collect?
• Email address – get this before you ask for
anything else to reduce drop-off
• Only ask for data that you will use
• On subsequent screens, ask for secondary and
tertiary information
– Demographics
– Interests
• Offer additional subscriptions
14

52. 9/18/2012
When to Act
• It’s a strategy, not a tactic
• Don’t wait until your existing list shrinks to
nothing to start thinking about acquisition
• Constantly have a line in the water
• Set multiple lines around key events
– Holidays if you are in retail
– Big PR announcements
– Other brand building campaigns
Doing it Right
A real world example – New York Magazine
15

53. 9/18/2012
Give yourself a few chances
Quick and to the point
16

54. 9/18/2012
Once you get a bite…
Expectations
Options
Go Social
17

55. 9/18/2012
Doing it Right
A real world example – Permission Data Co-Reg
Main Registration Path
18

56. 9/18/2012
Co-Registration Options
Welcome
Think of it as a first date
19

57. 9/18/2012
Now What?
• Once you get an email and consent, what should
you do?
20

58. 9/18/2012
Why are you yelling??
• This is a crucial time…
– Your subscriber is online
– You can direct them to their inbox to look for your
message
– Your brand is top of mind
– They are expecting to hear from you
And make sure it’s a 10
• Welcome messages typically enjoy great open
rates
• This is your only chance to make a first
impression
21

59. 9/18/2012
What to include
• Clear branding that will match the bulk of your
campaigns
• Well thought-out, concise copy
– Welcome them to your program
– Set expectations, especially for the first week
– Talk about value…
Membership has its privileges
• Your subscription better offer some value.
This is your first chance to tell them about it.
– Relevant content
– Relevant promotions
– Relevant updates
(See the theme???)
22

60. 9/18/2012
Goals for your Welcome Program
1. Establish your branding
2. Drive engagement
3. Promote your program’s value
4. Set expectations
5. Retention
6. Provide value
Doing it right
Some examples of what to do
23

61. 9/18/2012
24

62. 9/18/2012
25

63. 9/18/2012
Make it a Series
• Appropriate when you have a
lot of content – send smaller,
easily digestible pieces
• Repetition builds habits
• Multiple sends increases
likelihood of at least one open
• 3 to 5 messages over the first
week is not overwhelming
Promotion
In any investment, you expect to have fun
and make money.
-Michael Jordan
26

64. 9/18/2012
Why do we send email?
• The underlying goal of all email marketing
programs is the same…
Show me the Money
• Email is about providing good content…
– So people create more impressions on your site
• Email is about building our brand…
– So people buy more of your product
• Email is about providing value through
transactional messages…
– So your service is more valuable and people
remain customers and spend more money
27

65. 9/18/2012
What Makes it “Promotional”
• A message is considered promotional or
commercial if the message has a primary
purpose of advertising or promoting a
commercial product or service
Core Components - Header
• Assume that your from address and subject
line are the only parts of your message that
someone will see.
– Let them know what is inside
– Make it relevant
– Don’t waste characters overpersonalizing
– Don’t send from “noreply@yourdomain.com”
28

66. 9/18/2012
Doing it Right – Subject Lines
• 40% off pants tonight! Happy hours are here
again (Banana Republic)
• Dinner Out Tonight - $25 Gift Certificate for
Just $4 (RetailMeNot)
• Rory McIlroy Trusts Titleist to Capture
Deutsche Bank Championship (Titleist)
• Everyone wants one of these (HockeyMonkey)
• You deserve a bag you can’t live without
(eBags)
Not Quite There – Subject Lines
• Ideas for you
– Vague
– I have a feeling these are probably ideas for everyone,
not just me
• Big Savings on New Cars
– Sounds like a cars salesman…
• Damian, here are your top deals of the week
– Personalization adds nothing here
– Travel site should have said “Top Deals from Austin for
the week”
29

67. 9/18/2012
Core Components - Branding
• If it’s a 3rd party promotion, keep your
branding prevalent
• Keep the core offer as high on the screen as
possible
• Don’t have all your branding in images
Core Components - Offer
• Balance of text and images
• You have less than 5 seconds…make it obvious
• Strong Call To Action and several placements
• Create a sense of urgency
• Dazzle them with numbers (50% Off, 2 for 1)
30

68. 9/18/2012
Finding the Balance
• A great program has a balance between
promotional, content and transactional
messages
• Your subscribers will tell you where that
balance should be
• It is different for everyone
Doing it right
Some examples of what to do
31

69. 9/18/2012
Banana Republic
• Clear branding
• 40% offer jumps off
the page
• Links to web
• Clever theme
• Urgency
The New Yorker
• Good text to image
balance
• Pre-header
• Social sharing
featured
32

70. 9/18/2012
Southwest
• Southwest branding,
partner offer
• Tied to a transaction
• Subject identifies it
as a promotional
message
Transactional
Enabling everything else you want to do
while adding huge value to the subscriber
33

71. 9/18/2012
The Money Tree Does Exist
Transactional Email
• Look to HTML based
emails that provide
all relevant
information
• Transactional
messages have
+60% open rate and
can represent
significant revenue
34

72. 9/18/2012
20% promotional
content space
Amazon.com
• Gift Cards is the only
promotional message
– They pioneered the cross sell
35

73. 9/18/2012
Starbucks
Perfect
opportunity for
cross sell to coffee,
cups or gift
certificates
Norton/Symantec
Upgrade to a suite product
36

74. 9/18/2012
Costco
• Why not try to sell
me 5,000 asprin
McAfee
Text? Really?
37

75. 9/18/2012
Wine.com
No glasses, no similar wines
Transactional Based Email Programs
Recommendation
Product
Dynamic product
recommendation
s enhance both a
promotional and
Abandon
transactional
Cart
message
Strategy
NLP Triggers
Replenishment
Survey
38

76. 9/18/2012
Groupon
OMG – so
excited…for 5
seconds
Browse Remarketing
Stop with the Kitchen Sink
marketing tactics
39

77. 9/18/2012
Opt-Out
Let me out of here!
From the voice of the consumer
Why would
you
unsubscribe
from an email
list?
40

78. 9/18/2012
What are the primary reasons for
unsubscribing from an email list?
© Digital River Inc. , Source: BlueHornet Online Email Behavior Survey, Feb 2012 n=1,000
From the voice of the consumer
Could a company
change your mind
when you have
decided to opt-
out?
41

79. 9/18/2012
Would you opt-down if given the
option when unsubscribing?
© Digital River Inc. , Source: BlueHornet Online Email Behavior Survey, Feb 2012 n=1,000
Opt-Down
• Provides the
consumer a choice
of communication
based on common
reasons for attrition
– Frequency
– Relevancy
– Subscription Choice
42

80. 9/18/2012
Bonus: Win-Back Programs
We miss you!
43

81. 9/18/2012
What is a Win-Back Program
Win-Back programs
are inactive
customer
engagements
designed to re-
activate the
customer to an
active
communication
relationship
Why should I care?
The landscape has changed dramatically based on
reductions in force and changes in ISP business
models
• Rules for engagement
– Active engagement
– Reactive suppression of inactive
– Relevancy
44

82. 9/18/2012
Deliverability Risks
Low Risk Med Risk High Risk Critical Risk
Deliverability Risk
Low Risk
2012 2011 2010 2009 2008
As subscribers age, their emails change, especially
when we see inactivity. We recommend a short
window of engagement with consumer to protect
your overall deliverability.
Win-Back Programs
• Sample process is a series of 3 emails over a
short period of time
• Short term Win-Back can be 1 or 2 emails
45

83. 9/18/2012
Win-Back Programs
• Non-Offer Based
– Content sites
• Offer based
– Aggressive
– Limited duration
– Exclusive coupons to
prevent viral coupon use
– Segment that population
out of the normal stream
Putting it all Together
What to do next
46

84. 9/18/2012
Your To-Do List
1. Compare your acquisition program to your overall
program objectives
2. Make sure you take advantage of engaged subscribers
during that first week to ten days
3. Find the right balance of promotional messages
4. Take advantage of transactions to send highly engaging
messages
5. Be smart about keeping your list clean. Don’t send to
people who tell you they don’t want to hear from you.
6. Review all aspects of you program, all the time.
Contact
Damian Borichevsky
Damian@pulsepoint.com
Ryan Phelan
rphelan@digitalriver.com
47

85. 9/18/2012
Master The World of Email
Lists And Databases
Fine tune, take action and transform lists and
databases into high‐performance business
drivers
Agenda
State of Affairs
Acquiring Subscribers
The Quest for Data
Fine‐tuning Your List into a High Performance Business Driver
Maintaining a Healthy List
Monetizing your List Through Other Means
Closing: Resources, Exam Prep, Feedback
1

86. 9/18/2012
State of Affairs
• Email still rocks
• The shift from broadcast to automation
• “Inactives” are the #1 deliverability challenge
• Designing for mobile is no longer optional
• List rental is going old school
• Programmatic buying for email newsletter
advertising
Acquiring Subscribers
How can I add new prospects / customers to my database effectively?
Streamline the signup process
Best practices for permission
Set proper expectations
Acquisition channels and methods
2

87. 9/18/2012
Acquiring New Subscribers
Website registration is still #1
3

88. 9/18/2012
Three Methods To Increase Opt‐in
• The problem: There is a better way to introduce new users to
your service than imposing registration on them before they
get to the main features.
• Make registration easy, intuitive and valuable for the
customer.
• Give the customer control of how much information they
want to share with you.
• Start with basic questions and ask for deeper information
gradually.
Make Registration Easy
• Make the sign‐up box ubiquitous.
• Offer Social Connect:
– Facebook
– LinkedIn
– Twitter
– Yahoo
– WordPress
• Remove unnecessary fields
4

89. 9/18/2012
Registration Value Proposition
• Clearly explain the benefits
of registering and the value
that you intend to provide
through your email program
• Set expectations about what
the customer can expect
from your email program
• Make sure your emails
deliver on that promise
Best Practices for Permission
The old saying that double opt‐in is best may
be true for some, but for others it can be
harmful to growth
Create less friction by reducing the user’s
responsibility for permission, but be aware of
the cost.
5

90. 9/18/2012
Building Your List
Building Your List
• Growth Hacking (Virality)
– “If you create content that stirs emotions, causes laughter or inspires recipients
in some practical or meaningful way; they will probably want to share it with
others in their life." Tom O'Leary
• Buying has risks
• Go organic!
• Location, location, location
• Increase the value proposition of registering
• Customer acquisition methods and channels
– Focus on the audience
– Focus on the channel
– Focus on the creative and message
Bring it all together; a consistent message and creative elements targeted to
the right audience, through proper channels that set the right tone for your
brand.
6

91. 9/18/2012
The Quest for Data
Four types of data
Enhancing your data
Example: Welcome program with enhanced data
Preference centers
7

92. 9/18/2012
The Quest for Data
Segmentation is
primarily driven from
four types of data
Four Types of Data
• Endemic
- Data unique to a particular record
- e.g. B2B -Job title, company name, vertical industry; Demographic info
• Transactional
- Data captured during any transaction an opt-in has with your brand
- e.g. financial, opt-in landing page, subscriber’s IP address
• Behavioral
- Data captured during any interaction with subscriber
- e.g. website clicks, email clickthroughs, calls into technical service
• Computed
- Data is the outcome when one or more variables are used to create a
third variable
- e.g. The distance to subscriber’s home from your retail store, RFM
8

93. 9/18/2012
Enhancing Data
• Third-party profile append
• Progressive profiling tactics for
self-stated attributes
• Device and engagement
patterns from raw tracking data
• Advanced attribution analyses
Welcome Program With Enhanced Data
The Challenge
Customers are heavily engaged at the
onset of a subscription, a time when
marketers know the least about the
customer
The Solution
Leverage a real-time data append and
interactive content elements to better
understand new subscribers, creating a
more relevant and effective Welcome
experience
The Result
A relevant first-impression is delivered
to the customer creating a greater
likelihood of ongoing engagement long-
term
9

94. 9/18/2012
Welcome Program With Enhanced Data
Step 1 Step 2 Step 3 Step 4
Customer Subscribes Data is submitted and run real‐ All data elements are The first Welcome
time against an integrated third‐ passed into the Message in a defined
party database to overlay appropriate database series is triggered
available demo/psychographic
data elements
Step 5 Step 6 Step 7 Step 8
Content is generated An Interactive Poll will also be Upon response to the Remainder of series will be
dynamically based on the delivered within the content of the Poll, data submitted will automatically deployed – including
third‐party data provided ‐ if first message to collect additional be written back to content and poll questions
no match exists a default profile information about the customer database for dynamically served up and driven
communication will be sent customer. Questions will be future targeting by previous engagement or lack
dynamically generated and driven by thereof
presence or lack of overlay data
Preference Centers
In addition to demographics
• Drive targeted communications with this and attitudes, include:
self‐reported menu of “likes” and “dislikes” • A way to declare communication
preferences: type, frequency,
• Only ask for information you plan to use device (e.g., mobile-friendly)
• A clear and easy way to
unsubscribe (or subscribe) from
• Determine how much control you can give all mailings
recipients without offering an • A mechanism for password
change
unmanageable number of preferences • A mechanism for email change
of address
• Evolve preference options along with your
segmentation opportunities
• Occasionally ask customers to refresh their preferences, as
preferences change over time
10

95. 9/18/2012
Fine-tuning Your List into a High
Performance Business Driver
Success starts with segmentation
Popular segmentation strategies
Tailoring to the customer lifecycle
Triggered emails
Individual exercise
Success Starts with Segmentation
The good news is most marketers realize not all subscribers are alike
Enjoy reading Evaluate
what the CEO Like learning
purchasing
I look forward thinks money saving I recently
options
to coupons tips changed jobs
11

96. 9/18/2012
Popular Segmentation Strategies
Use segments to
– Understand how
different groups within
your email database
respond
– Provide your audiences
with appropriate content
and offers
Case Study
Challenge
Needed to send 2 million emails/day with
precisely targeted promotions
Solution
Improved email targeting with data mining
for more customized and personalized emails
Result
Increased from 3‐5 sales events per day to
18 per day across 7 categories based upon
subscriber engagement
7% increase in subscriber engagement with 12% increase in open rates and 25%
increase in click‐through rates
12

97. 9/18/2012
Case Study
Campaign Objective
Drive member engagement and purchase
conversion.
Campaign Strategy
Targeted email that notifies members that
their favorite brand is back, two hours
before event launches.
Key Metrics Tracked
Open Rates, Click‐through rates, Purchase
Conversion, Revenue/Email Delivered.
Case Study
Description
Triggered email alerting members when their
favorite brands are back on HauteLook.
Goal
Drive purchase/repeat purchase and
engagement.
Target Audience
Members who have purchased or carted an
item from targeted brand.
Frequency ‐ Daily
Why it works
• One‐to‐one marketing. It doesn’t get much
more relevant than this!
13

98. 9/18/2012
Case Study
Brand vs. Category Level Targeting
Results 343%
Compared to category‐level targeted
emails, brand affinity emails drive:
• 52% more opens 216%
• 32% more clicks
• 2x more purchases/click
• 3.5X higher revenue/email delivered
52%
32%
Opens Clicks Click to Purchase Rev/Email Delivered
What’s Next
Scale and automate this campaign!
Tailoring to the Customer Lifecycle
Acquisition Conversion Growth Retention Reactivation
1. Email
Appends
2. Preference
Centers 10 12
11
3. Subscription 8 9 14
13
Management 15
6
7
4. Mobile
5
Acquisition 17
16
5. Welcome 9. Welcome
4
3 Programs Programs
12. Editorial 15. Reactivation
6. Shopping Cart 10. Enhanced Wrapping Programs
1 2 Abandonment Transactional
Email 13. Lifecycle 16. List Hygiene
7. Landing Pages Programs
11. Sharing to 17. Win-back
8. Click-stream Social 14. Surveys Programs
Triggered Networks
Messages
14

99. 9/18/2012
Case Study
Challenge
Mature email list with 60%+ unengaged
(no opens, clicks, site browses) in last 12
months
Solution
Target unengaged segment with enticing
subject line and compelling offer
Test
Subject Line A:
“Save an additional 10% for a limited
time only.”
Case Study
Results
• Customers responded to a valued
customer message over generic
• Subject line test generated a 10%
lift
• A 15% offer increased CTO ratio by
2.3%
What’s Next
Scaled and automated ‐‐ Win‐Back
has become a regular at Travelocity
15

100. 9/18/2012
Triggered Emails
A single or series of emails created
by a defined set of rules based on
dates, events or behaviors that are
automatically sent to a subscriber
Consistently achieve higher open
rates due to timing and relevancy
Popular Triggered Campaigns
Welcome 63%
Thank you 54%
Transactional (e.g. bills, receipts) 48%
Post purchase (e.g. product review) 32%
Upsell (e.g. product recommendations) 26%
63% of marketers send
Activation (e.g. how to) 24% automated welcome
Date triggered (e.g. renewals, reorder) 22%
email messages
Win back/ reengagement 17%
Event countdown 16%
Triggered based on website behavior 13%
Shopping cart abandonment 11%
Source: ©2011 MarketingSherpa Email Marketing Benchmark Survey
Methodology: Fielded July 2011, N=2,735
16

101. 9/18/2012
Effectiveness of Relevancy Improving Tactics
Triggered emails
are the top
relevancy tactic
Case Study
Challenge
Wanted to convert Active Shoppers to
“Purchasers” by sending triggered low
fare alerts for price drops, targeting
consumers who previously shopped
but didn’t book their trip
Solution
• Leverage triggered‐alerts
• Feature relevant offers based on a
specific city pair and date combination
• Integrated relevant cross‐sell offers
with built‐in search functionality
17

102. 9/18/2012
Case Study
Results
While a small fraction of overall email
volume, this email drives 12% of
overall bookings
2‐3x of average email open and click‐
through rates
Low Fare Alert generates conversions
that are 300% ‐ 400% higher than
other conversion programs
Individual Exercise
• List data points you store that could help drive segmentation
• Write down 3 new segments your company is not targeting today
and develop a high‐level plan for how you would target them
• Consider:
– Demographics
– Geography
– Psychographics
– Business and Behavioral History, including Lifecycle
– Trigger opportunities
18

103. 9/18/2012
Maintaining a Healthy List
What is a healthy list?
Is your list healthy?
Methods to improve your list health
Case study
Less Email is Being Delivered (Inbox)
ReturnPath, The Global Email Deliverability Benchmark Report, 2H 2011
19

104. 9/18/2012
State of Affairs
ReturnPath, The Global Email Deliverability Benchmark Report, 2H 2011
What is a Healthy List
20

105. 9/18/2012
The three indicators of a healthy list
Revenue Engagement Growth
Is My List Healthy?
Focus on Engagement
• Inbox Placement Rate (IPR) is the rate at which the ISPs
deliver the mail received from you to their customers’
mailboxes
• IPR is determined by your sender reputation, which can
largely be measured at SenderScore.org
21

106. 9/18/2012
It’s all About Sender Reputation...
* Silverpop & ReturnPath Keynote, 2012
According To Return Path..
Senders with better reputation metrics have cleaner lists.
Unknown User rates above 1% for legitimate email is considered
problematic.
Return Path, The Sender Reputation Report, 2011
22

107. 9/18/2012
More Fun Metrics
• Average Complaint Rate: <0.18
• Average Open Rate: 8%
• Average List Depreciation: 20% annually
• IP reputation and how to measure
– SenderScore.org
– Senderbase.org
How to improve the health of a list
23

108. 9/18/2012
List Hygiene
List Hygiene
• The Unengaged
• Send more mail. Like rain, the more you mail the
cleaner your list becomes. But at what cost?
• Sign up for feedback loops and remove complaints
and hard bounces
• 3rd party validation and hygiene
24

109. 9/18/2012
Managing Preferences
Consider opt‐down
options to let
subscribers put your
emails on pause
List Hygiene – Example
BACKGROUND
• Major company in the publishing industry who sends1‐3 million emails a day
• Regular (bad) habit of purchasing lists
• Ended up with 40% bounce rate which triggered a spamhouse listing as
someone who buys lists or does appending
• ISP's decide how they want to deal with you but most ISP's use the spamhouse SDL
listing to just block you outright... even internationally
IMPACT
• Blocked!
– 30% of hotmail blocked
– 75% of yahoo blocked
– Also blocked at comcast and gmail
• Spent a lot of time and money remediating these issues
– Cut back frequency to unengaged subscribers and took 8 months to clean it up
THE BOTTOM LINE
• Spam is about 80‐90% of emails sent. Purchased address have the same problems
as a piece of spam ‐‐ the email was not requested.
• If you end up on a Spamhouse listing, even those recipients who DO want email
from you will not get your messages.
• Easier to start off right than try to get back to right.
25

110. 9/18/2012
Monetizing your List
The Email Advertising Opportunity
Email Ad Opportunity
26

111. 9/18/2012
Newsletter
Ads
Dedicated
List Rental
Offer
27

112. 9/18/2012
Hybrid
Newsletter
/ Dedicated
Other Monetization Methods
• Monetize data assets
– Connect audience intelligence across channels – Online/Offline/cross‐property
data aggregation, segmentation, etc.
– Monetize data assets any way you want – you sell, we sell, public, custom,
private, etc.
– Provide seamless media integration for data activation
– Access to marketers/buyers
• Affiliate links
• Facebook retargeting ads?
28

113. 9/18/2012
Contact
Amanda Hinkle
Strategy Team Lead, StrongMail
ahinkle@strongmail.com
650‐421‐7125
Craig Swerdloff
Founder & CEO, LeadSpend
cswerdloff@leadspend.com
917‐648‐6838
@swerd
Exam Questions
29

114. 9/18/2012
Acquiring Subscribers
• What is the #1 most effective method to drive list growth?
• What is more effective, incented or non‐incented site
registration for driving list growth?
• Does Facebook Social Connect provide email by default?
• What is the #1 most important disclosure at signup?
• True or false: Collect as much information as you can during
the email signup process so you can more effectively target
your communications moving forward
– False
• Name a common email permission type
– Any one of the following will work: Single, Double, Pending
The Quest For Data
• Name the four classifications of data discussed
in the class
– Endemic, behavioral, transactional, computed
• Who is the regulatory body governing CAN‐
SPAM?
– The FTC
30

115. 9/18/2012
Fine‐Tuning Your List
• The majority of organizations report being able to
segment subscriber data based on what transactional
data type?
– Purchase history
• Name the five stages in the customer lifecycle
– Acquisition, conversion, growth, retention, reactivation
• Name the type of message that is created by a defined
set of rules based on dates, events or behaviors and
are automatically sent to a subscriber.
– Triggered
• What is the most popular type of triggered campaign?
– Welcome
Maintaining a Healthy list
• True or False: Immediately remove individuals from your email file if they have
not clicked a message in the past 12 months.
– False
• According to Return Path, what % of mail is being delivered into the inbox?
– 25%, 75%, 50%, 100%?
• How much does the average list depreciate in size per year?
• What % of “unknown users” or hard bounces does Return Path say is
problematic?
• What the average rate of complaints for senders?
– 2%, 1.6%, 0.16%, or 2.6%
• What is the generally accepted average open rate?
– 5%, 10%, 8%, 12%
• Fill in the blank: Opt‐down is an alternative you could offer to a customer
opting ___.
– out
31

116. 9/18/2012
Monetizing your List
• According to Return Path, what % of mail is being
delivered into the inbox?
– 25%, 75%, 50%, 100%?
• How much does the average list depreciate in size per
year?
• What % of “unknown users” or hard bounces does
Return Path say is problematic?
• What the average rate of complaints for senders?
– 2%, 1.6%, 0.16%, or 2.6%
• What is the generally accepted average open rate?
– 5%, 10%, 8%, 12%
32

117. 9/18/2012
Email Marketing
Part 4: Produce Engaging
Creative, Every Time
1
In this session you will learn:
How to use a foundation email best practice
principles to craft the most engaging content for your
email program, campaign and individual message.
Using these guidelines will ensure optimal
performance for your email, whether your audience is
interacting with your message in a traditional desktop
environment or an on-the-go mobile device.
dma2012.org
1

118. 9/18/2012
In this session we will cover:
Creative Effective
Best Practices Email Content
• Understanding the Channel • Layout
• Defensive Design • Animation
• Copywriting Tips • Video
• The Mobile Inbox • Teasers
• Dialogue Starters
• Cross-Channel Tips
dma2012.org
The Email Channel
What makes good email creative?
2

119. 9/18/2012
Acknowledge
the Channel
• What works for print does not
translate well into email
• Recipients don’t often view the email
in its entirety, they scroll through
micro experiences and scan copy
looking for highlights
• Print relies heavily on imagery, email Magazine Layout
can’t always count on that creative
element to tell its story
• Size matters, emails download
progressively, viewers miss out on
content in heavy files
Direct Mail Postcard
dma2012.org 5
The principles of
web design often apply
to email, but with
notable exceptions
• Email clients are very restrictive
and lack standards, rendering fidelity
is a challenge
• Rich media including forms,
scripting and Flash won’t work
properly and can effect deliverability
• Don’t worry! We will cover what you
can do soon
Website
dma2012.org 6
3

120. 9/18/2012
The Inbox Mindset
Viewers will take 2-5 seconds to Words to work by
decide if the email message is relevant An email is like a store
window; it needs to reveal just
enough to compel visitors to
• Interacting with an email is different from browsing online
enter and explore
• Viewers aren’t necessarily looking for something
• Email recipients need to be engaged quickly and be pulled
vigorously through the content
dma2012.org 7
Embrace email best practices, otherwise...
you design this…
v
but your audience
experiences this!
dma2012.org 8
4

121. 9/18/2012
Let’s talk about three key email
best practice topics:
1. Defensive Design
2. Copywriting
3. The Mobile Inbox
dma2012.org 9
Defensive Design
5

122. 9/18/2012
Default Image Suppression
The trend towards disabled images is growing
email client default images on? default links on? default changeable?
AOL 9.0 software no no yes
AOL.com, AIM.com web yes yes yes
Gmail web no yes yes
Hotmail web yes yes yes
Lotus Notes software yes yes yes
.Mac web yes yes no
Outlook 2009, 2010 software no yes yes
Yahoo web no yes yes
dma2012.org 11
Optimize for the Preview Pane
• 9 of 10 users have a preview pane
• 7 of 10 say they frequently or always use
it
• Up to 60% of desktop viewers have
images blocked and the trend is growing
• Up to 50% of users don’t make a habit of
scrolling
• People generally read right left to right,
top to bottom
• The reading order is typically headline,
bullet, CTA, then paragraph copy
• Remember, mobile devices clip heavy
files
dma2012.org 12
6

123. 9/18/2012
Optimize for the Preview Pane
Every recipient should be able to
answer the following three questions
whether or not images are blocked:
• WHO are you?
• WHAT do you want me to do?
• WHY should I do that?
dma2012.org 13
Smart Work Around Tips
Three ways to beat disabled images
1. Use HTML instead of graphical text
2. Include preheader and alt tags
3. Employ the Bulletproof Button
Images disabled
Images disabled
Images enabled
Images enabled
dma2012.org 14
7

124. 9/18/2012
Optimize for the (Mobile) Preview Pane
• Many of the most popular
mobile devices display
images by default
• You must answer the who,
what & why question in a
smaller space
• Heavy messages can be
clipped
dma2012.org 15
Email Anatomy PREHEADER
HEADER
NAVIGATION
• Preheader
TABLE OF CONTENTS
• Header
• Navigation
• Table of Contents
(preview pane banner)
PRIMARY MESSAGE
• Primary message
• Submessage(s)
• Side rail
• Recovery module
• Footer SECONDARY MESSAGE
TERTIARY MESSAGE
RECOVERY MODULE
FOOTER
dma2012.org 16
8

125. 9/18/2012
Email Anatomy
• Snippet text appears in Outlook and Gmail
• Only 21% of major retailers have optimized their email for snippet
dma2012.org 17
Email Anatomy
• Preheader
• Header
• Navigation
• Primary message
• Submessage(s)
• Recovery module
• Footer
dma2012.org 18
9

126. 9/18/2012
Email Anatomy
Mapping a hierarchy is a great place to start, but don’t stop there!
Use it as a framework for compelling and creative content.
dma2012.org 19
Copywriting
10

127. 9/18/2012
Subject Lines
Subject line best practices can vary depending
on the audience, test and refine based on your
audience’s response
• Optimize length for limited preview space
• Be specific about the content in the email
• Test what works
– Personalization
– Brand name
– Mention product and/or category
– Benefit of opening email
– Urgency / deadlines
– “Teasers” that tempt subscribers to open
– Call out big savings
dma2012.org 21
Craft Effective Copy
• Headlines should be short and punchy
• Use bullet points for maximum scannable
• Include action language within body copy,
and hyperlink those words or phases
• Take the landing experience into account,
your conversion metrics will thank you
dma2012.org 22
11

128. 9/18/2012
Calls-to-Action
The 3c’s of calls-to-action:
1. Clear
2. Concise
3. Clickable
• Keep it short (10-20 characters)
• Create urgency with words like “Now”
• Set expectations: tell the user where
they’ll be taken whey they click
• Use specific action verbs
dma2012.org 23
Copywriting Make-Over
Longchamp Your Very Own Le
before
Pliage
Longchamp’s famous foldable nylon bag
is now completely customizable. Take
your pick of sizes from a petite clutch to
a hefty weekender, then choose the
colors, handles, hardware, and, of
course, the monogram. Comes in 15
colors.
Your Longchamp®, Your Style after
Now Longchamp’s famous foldable nylon bag
Now Longchamp’s famous foldable nylon bag
is completely customizable
is completely customizable
• Choose from petite clutches to
a hefty weekenders
• Pick your own handles & hardware
• Select from one of 15 colors
• Don’t forget your monogram
Design Your Bag Now
Design Your Bag Now
email sample
dma2012.org 24
12

129. 9/18/2012
The Mobile Inbox
Mobile Optimization
- Shorten subject lines
- Set scale to screen
- Streamline content
- Think “fat fingers”,
enlarge fonts & add
whitespace to clickable
regions
- Consider one column
layout view
- Increase color palate
contrast
dma2012.org 26
13

130. 9/18/2012
Responsive Layout
• Responsive layout is a
new approach to the mobile
inbox conundrum
• Using the CSS rule ‘@media’
to read screen dimensions we
are able send one file but
display a unique layout for
each experience
• This technique works
best on native mail apps for
each device
Desktop View Mobile Device View
dma2012.org 27
Best Practices Summary
• Optimize the preview pane payload, answer
the three W’s without relying on images
• Each email needs a clear hierarchy, map the
anatomy of the message before diving into
design and copy
• Copy be scannable, remember the 3C’s of
calls-to-action
• Consider the mobile experience, edit for the
small screen
dma2012.org
14

131. 9/18/2012
Now that we understand the
foundation of email best practices,
let’s learn how to make a message
that activates your audience!
dma2012.org 29
Effective Content
Crafting engaging creative
15

132. 9/18/2012
The “S” Curve Layout
• Lay the groundwork for the
next scroll
• Use image placement to
create dynamic flow
• Draw viewers through micro
experiences with a flowing
curved layout
dma2012.org 31
Horizontal Scroll Layout
• Horizontal emails are perfect for mobile touch
screen interaction
• Crop content to encourage exploration
• Panoramic layouts match well with the luxury
brand experience
dma2012.org 32
16

133. 9/18/2012
Modular Content Layout
• Template modules allow you to test content
dma2012.org 33
Animation
• Simple and easy way to
stand out in the inbox
• Gifs supported by nearly
every email client
• Creates a richer experience
and drives engagement
dma2012.org 34
17

134. 9/18/2012
Animation Accents
• Draw attention to important details, such as offers and gift guides
• Act as small accent to delight users, great with seasonal themes
dma2012.org 35
Animation as Feature
• Motion can tell a story
quickly
• Jazz up simple promos with
a fun theme
• Compliment the overall
brand experience
dma2012.org 36
18

135. 9/18/2012
Animation for Personalization
• Personalization is a great
way to pop out in the inbox
• Easy way to convey a benefit
• Increases the surprise and
delight factor
dma2012.org 37
Video Spoofs
• Demonstrate products
• Instruct viewers
• Preview web content
dma2012.org 38
19

136. 9/18/2012
Video Simulations
• Effective way to use the power of
motion without the worry of
deliverability and play back issues
• Create a memorable brand
experience that stands out
• Kick off a media rich
cross-channel campaign
dma2012.org 39
Embedded Video
• Send video directly to
the inbox
• Unless protocols are
set properly the
experience may suffer
• Several services help
facilitate the
compressions and
delivery of video to the
inbox
dma2012.org 40
20

137. 9/18/2012
Teasers
• Interrupt the normal flow
of an email
• User teaser content to
prompt exploration
• Try a unique take on
brand voice
dma2012.org 41
Dialogue Drivers
• Opportunities to share
• Surveys and polls
• Request for feedback or ratings
• Customer reviews and quotes
dma2012.org 42
21

138. 9/18/2012
Cross-Channel Tip
• Email can be a great promotional tool for
your social acquisition and lifecycle goals
• Social can be a great channel for
research for your email content
– What’s most active on your Facebook page,
Twitter feed, YouTube channel or Pinterest
boards?
– Social channels are an easy way to gauge
interest in a topic
– Look at your customer’s behaviors in social to
customize email content per user or segment
• Remember that your email exists in a
larger social landscape, make it part of
the conversation
dma2012.org 43
Content Summary
• Ensure that your message is built on a solid
best practice foundation
• Email is the beginning, content should drive
to next step
• Motion drives engagement, test different
approaches to find the best response rate
• Use dialogue drivers to open up the
conversation
• Leverage the social channels to read your
audience and draw content assets
dma2012.org
22

139. 9/18/2012
Bonus Round!
dma2012.org 45
Email Creative Assessment
Let’s judge some emails!
23

140. 9/18/2012
Email Creative Assessment
PREVIEW
PANE
PREVIEW
PANE
Non-Optimized Creative Optimized Creative
dma2012.org 47
Email Creative Assessment
PREVIEW
PANE
PREVIEW
PANE
dma2012.org 48
24

141. 9/18/2012
Email Creative Assessment
iPhone iOS scales Android OS iPhone iOS
content smartphone –
top left view &
‘Images on’
Non-Optimized Optimized
dma2012.org 49
Email Creative Assessment
Compare and Contrast
• Clarity of message
• Strength of calls to
action
• Use of whitespace
• Message hierarchy
dma2012.org 50
25

142. 9/18/2012
Thank You!
Lynn Baus
Creative Director, Responsys
lbaus@responsys.com
@lynnbausmktg
26

143. 9/18/2012
The Art and Science of
Email Testing
1

144. 9/18/2012
Training Goals
Agenda
• Getting started
• Setting up a test
• What to test
• Common testing pitfalls
• Tools for testing
• Wrap up
2

145. 9/18/2012
Why Test?
• Gain insights quickly
• Measureable
• Learn and improve
• More ROI!
7 Steps to Getting Started
How to establish the foundation for a successful testing program
3

146. 9/18/2012
#1 Get Executive Buy-in
#2 Make it a Team Effort
4

147. 9/18/2012
#3 Gather Data
• Analytics platform
• Social Media sites
• Search keywords
• Customer data fields available
• Your own knowledge
• Teammates
#4 Establish the Infrastructure
5

148. 9/18/2012
#5 Track and Share Results
#6 It’s OK to Miss the Mark
6

149. 9/18/2012
#7 There is no such thing
as a bad test
…..it needs to be
implemented correctly.
7 Steps to Getting Started
1. Get Executive Buy‐in
2. Make it a Team Effort
3. Gather Data
4. Establish the Infrastructure
5. Track and Share Results
6. It’s Ok to Miss the Mark
7. There is no such thing as a bad test
7

150. 9/18/2012
Setting Up a Test
How to set up an email test correctly
Determine Your Objective
8

151. 9/18/2012
Establish a Hypothesis
Define the Success Metric
9