The Home Energy Saver is an action-oriented home energy benchmarking site. The do-it-yourself home energy assessment helps consumers understand and streamline their home energy use through behavioral and structural upgrade recommendations. We present a longitudinal case study describing how we used web-analytics to iteratively benchmark, track and improve the persuasive pull of the site.

1. NOT YOUR FATHERʼS
WEB ANALYTICS REPORT#
David Mahaffey!
!
!
Kath Straub!

2. SPECIAL THANKS TO …#
Diane Chojnowski! Evan Mills!
Social Media Strategist! Staff Scientist
Usability.org! Berkeley Labs!

3. Practitioner takeaways!
• Define measurable success metrics early!
• Build clear and meaningful KPIs!
• Team with/make use of analytic practitioners!
• Side step the data blizzard!
#
#
Next generation AX#
Improve the user journey in real time!

4. Roadmap!
• Practices UX and AX!
• AX tools are difficult!
• Visitor life cycle!
• An example from Home Energy Pros!
• Metrics definitions!
• Persuasion in a Newsletter!
• Persuasion API - Using AX in real time!
• Summary!

5. Practices have complementary processes!
Build
KPIs
Collect
Data
Define
Goals
Analyze
Data
Analy:cs
Process
Implement
Test
Alterna:ves

6. Practices have complementary processes!
Build
KPIs
Collect
Data
Define
Goals
Analyze
Data
Analy:cs
Process
Implement
Test
Alterna:ves
Create
Mock-­‐ups
Usability
Tes:ng
Storyboards
Usability
Process
Define
Goals
Analyze
Data
Implement
Revise
and
Retest

7. Practices have common goals!
UX
goals
AX
goals
Understand
describe
why
people
do
what
Show
what
people
actually
do
(sort
of)
they
do

8. Practices have common goals!
UX
goals
AX
goals
Understand
describe
why
people
do
what
Show
what
people
actually
do
(sort
of)
they
do
Con:nuous
improvement
Goal
con:nuous
improvement
requires:
• Investment
in
the
process
• A
corporate
champion
Persuade
Explain
and
influence
behavior
toward
Measure
effec:veness
of
site
design
business
goals

9. Timeline UX and AX!
1980
1990
2000
2010
Internet
WWW
Web
2.0
Events/technology
related
to
UX
What’s
happening
in
prac:ce
of
UX
Events/technology
related
to
AX
What’s
happening
in
prac:ce
of
AX

10. Timeline UX and AX!
1980
1990
2000
2010
Internet
WWW
Web
2.0
Events/technology
related
to
UX
UX
exist
in
desktop
world
/
un-­‐networked
What’s
happening
in
prac:ce
of
UX
UX
in
so[ware
Cos.
w/
developers
Events/technology
related
to
AX
What’s
happening
in
prac:ce
of
AX

11. Timeline UX and AX!
1980
1990
2000
2010
Internet
WWW
Web
2.0
Events/technology
related
to
UX
UX
exist
in
desktop
HTML
-­‐
lots
of
world
/
un-­‐networked
sites
no
UX
What’s
happening
in
prac:ce
of
UX
UX
in
so[ware
Cos.
w/
developers
Events/technology
related
to
AX
Log
files
What’s
happening
in
prac:ce
of
AX

12. Timeline UX and AX!
1980
1990
2000
2010
Internet
WWW
Web
2.0
Events/technology
related
to
UX
UX
exist
in
desktop
HTML
-­‐
lots
of
e-­‐Commerce
need
for
world
/
un-­‐networked
sites
no
UX
UX
heats
up
What’s
happening
in
prac:ce
of
UX
UX
in
various
web
development
orgs
UX
in
so[ware
Cos.
w/
developers
Events/technology
related
to
AX
Log
files
Proto
web
tools
What’s
happening
in
prac:ce
of
AX
AX
owned
by
IT

13. Timeline UX and AX!
1980
1990
2000
2010
Internet
WWW
Web
2.0
Events/technology
related
to
UX
UX
exist
in
desktop
HTML
-­‐
lots
of
e-­‐Commerce
need
for
world
/
un-­‐networked
sites
no
UX
UX
heats
up
What’s
happening
in
prac:ce
of
UX
UX
in
various
web
development
orgs
UX
in
so[ware
Cos.
w/
UX
begins
rela:onship
w/
developers
marke:ng
Events/technology
related
to
AX
Log
files
Proto
web
Beginning
of
current
tools
web
tools
1.0
What’s
happening
in
prac:ce
of
AX
AX
owned
by
IT
Marke:ng
has
approach/
avoidance
rela:onship
w/
AX

14. Where do these practices live, who do they serve?!
IT
Marke:ng
Both
prac:ces
are
most
associated
with
IT
Both
prac:ces
goals
serve
marke:ng
That
is
where
the
building
and
implementa:on
Audience
of
interest
for
data
is
marke:ng
• UX
historically
wanted
to
influence
• Focus
groups
don’t
work
well
for
assessing
• AX
needed
people
with
these
skills
web
site
design/effec:veness
• UX
wants
to
know
why
goals
are
not
met
• AX
focus
on
what
are
people
doing
on
site
(sort
of)
Marke:ng
has
approach/avoidance
rela:onship
with
AX

15. Where do these practices live, who do they serve?!

16. Roadmap!
• Practices UX and AX!
• AX tools are difficult!
• Visitor life cycle!
• An example from Home Energy Pros!
• Metrics definitions!
• Persuasion in a Newsletter!
• Persuasion API – Using AX in real time!
• Summary!

17. Analytic tools produce data and reports!

18. Analytic tools produce data and reports!

19. Analytic tools produce data and reports!

20. Analytic tools produce data and reports!

21. Analytic tools produce data and reports!

22. Analytic tools produce data and reports!
Many
reports.
Li_le
ac:onable
informa:on
...

24. Roadmap!
• Practices UX and AX!
• AX tools are difficult!
• Visitor life cycle!
• An example from Home Energy Pros!
• Metrics definitions!
• Persuasion in a Newsletter!
• Persuasion API - Using AX in real time!
• Summary!

25. Visitor life-cycle – Behaviors of interest!
1. Enter
Capture
Entry
Page

26. Visitor life-cycle – Behaviors of interest!
1. Enter
2. Explore
Capture
Entry
Engaged
Browse
&
Page
Explore
Unengaged

27. Visitor life-cycle – Behaviors of interest!
1. Enter
2. Explore > Persuasion Opportunity
Capture
Entry
Engaged
Browse
&
Persuasion
Page
Explore
opportunity
Unengaged

28. Visitor life-cycle – Behaviors of interest!
1. Enter
2. Explore > Persuasion Opportunity
3. Enter Conversion Path
Capture
Entry
Engaged
Browse
&
Persuasion
Conversion
Path
Begin
Page
Explore
opportunity
Conversion
Unengaged
Unpersuaded

29. Overview of the visitor life-cycle
1. Enter
2. Explore > Persuasion Opportunity
3. Enter Conversion Path > Commit/Convert
!
Capture
Entry
Engaged
Browse
&
Persuasion
Conversion
Path
Begin
Commit
to
Converted
Page
Explore
opportunity
Conversion
Convert
Unengaged
Unpersuaded
Unconverted

30. Key transition points in the visitor life-cycle!
1. Capture
2. Engagement
3. Persuasion
4. Conversion

31. Key transition points in the visitor life-cycle!
1. Capture
2. Engagement
3. Persuasion
4. Conversion
Capture
Entry
Engaged
Browse
&
Persuasion
Conversion
Path
Begin
Commit
to
Converted
Page
Explore
opportunity
Conversion
Convert
Unengaged
Unpersuaded
Unconverted

32. Key transition points in the visitor life-cycle!
1. Capture
2. Engagement
3. Persuasion
4. Conversion
Capture
Entry
Engaged
Browse
&
Persuasion
Conversion
Path
Begin
Commit
to
Converted
Page
Explore
opportunity
Conversion
Convert
Unengaged
Unpersuaded
Unconverted

33. Key transition points in the visitor life-cycle!
1. Capture
2. Engagement
3. Persuasion
4. Conversion
Capture
Entry
Engaged
Browse
&
Persuasion
Conversion
Path
Begin
Commit
to
Converted
Page
Explore
opportunity
Conversion
Convert
Unengaged
Unpersuaded
Unconverted

34. Key transition points in the visitor life-cycle!
1. Capture
2. Engagement
3. Persuasion
4. Conversion
Capture
Entry
Engaged
Browse
&
Persuasion
Conversion
Path
Begin
Commit
to
Converted
Page
Explore
opportunity
Conversion
Convert
Unengaged
Unpersuaded
Unconverted

35. Roadmap!
• Practices UX and AX!
• AX tools are difficult!
• Visitor life cycle!
• An example from Home Energy Pros!
• Metrics definitions!
• Persuasion in a Newsletter!
• Persuasion API - Using AX in real time!
• Summary!

36. Home Energy Saver Suite!
For
Consumers
For
Home
Energy
Professionals
Home
Energy
Home
Energy
Home
Energy
Social
Community
Efficiency
Efficiency
Scoring
Tool
Site
for
Home
Energy
Calculator
Calculator
(DOE-­‐ARRA)
Professionals
(NING)

37. Home Energy Saver Suite!
For
Consumers
For
Home
Energy
Professionals
Home
Energy
Home
Energy
Home
Energy
Social
Community
Efficiency
Efficiency
Scoring
Tool
Site
for
Home
Energy
Calculator
Calculator
(DOE-­‐ARRA)
Professionals
(NING)

38. Home Energy Pros: Capture opportunity

39. Home Energy Pros: Capture opportunity
Callouts
Capture
Entry
Page

40. Home Energy Pros: Engaged visit
Callouts
for
more
engagement
opportuni:es
Capture
Engaged
Entry
Browse
&
Page
Explore
Unengaged

41. Home Energy Pros: Engaged visit
Engagement
/
persuasion
opportuni:es
Capture
Entry
Engaged
Browse
&
Persuasion
Page
Explore
opportunity
Unengaged
Unpersuaded

42. Home Energy Pros: Committed visit
Begin
conversion
path

43. Home Energy Pros: Committed visit
Begin
conversion
path
Capture
Entry
Engaged
Browse
&
Persuasion
Conversion
Path
Begin
Page
Explore
opportunity
Conversion
Unengaged
Unpersuaded

44. Home Energy Pros: Committed visit
Con:nue
the
conversion
path
Capture
Entry
Engaged
Browse
&
Persuasion
Conversion
Path
Begin
Commit
to
Page
Explore
opportunity
Conversion
Convert
Unengaged
Unpersuaded
Unconverted

45. Home Energy Pros: Converted visit
Complete
conversion
path
Capture
Entry
Engaged
Browse
&
Persuasion
Conversion
Path
Begin
Commit
to
Converted
Page
Explore
opportunity
Conversion
Convert
Unengaged
Unpersuaded
Unconverted

46. Roadmap!
• Practices UX and AX!
• AX tools are difficult!
• Visitor life cycle!
• An example from Home Energy Pros!
• Metrics definitions!
• Persuasion in a Newsletter!
• Persuasion API – Using AX in real time!
• Summary!

48. Metrics definitions
Metrics are divided into two parts!
1. Conversion!
– Traditional conversion!
– Engaged conversion!
– Committed conversion!
2. Indices of engagement!
– Persuasion index!
• Pages are associated with conversion opportunities !
– Contribution index !
• The contribution of each page to a conversion!

49. Metric definitions: Conversion types
• Tradi:onal
conversion
• Engaged
conversion
• Commi_ed
conversion
Sales
Learn
visitor’s
interests
Networking
Tradi&onal
Conversion
Loyalty
Engaged
Conversion
Commi2ed
Conversion
Site
Visits
Browse Choose a
Start
Commit
to
Converted
Enter
Site
& conversion
Conversion
Convert
Explore option Visitor
Unengaged
Unpersuaded
Unconverted
Tradi:onal
Conversion
Engaged
Conversion
Commi_ed
Conversion
=Convα/Vα
=Convα/Vα-­‐VU
=Convw/inpath
/Vw/inpath

50. Metric definitions: Traditional conversion
• Tradi&onal
conversion
=
Total
conversions
Total
site
visits
Tradi&onal
Conversion
Site
Browse Choose a
Visits
Enter
Site
& conversion
Start
Commit
to
Converted
Conversion
Convert
Explore option Visitor
Unengaged
Unpersuaded
Tradi:onal
Conversion
=Convα/Vα

51. Metric definitions: Engaged conversion
• Engaged
conversion
=
Total
conversions
Total
site
visits
-­‐
bounce
Tradi&onal
Conversion
Engaged
Conversion
Site
Browse Choose a
Visits
Enter
Site
& conversion
Start
Commit
to
Converted
Conversion
Convert
Explore option Visitor
Unengaged
Unpersuaded
Tradi:onal
Conversion
Engaged
Conversion
=Convα/Vα
=Convα/Vα-­‐VU

52. Metric definitions: Committed conversion
• Commi3ed
conversion
=
Total
sub-­‐type
conversions
Registra:on
Total
visits
to
that
sub-­‐type s
•
Invite
a
priend
v
ideo
conversion
page(s)
•
f
• Upload
icture,
• Post
comment
or
blog
• Print
or
download
from
site
Tradi&onal
Conversion
Engaged
Conversion
Commi2ed
Conversion
Site
Browse Choose a
Visits
Enter
Site
& conversion
Start
Commit
to
Converted
Conversion
Convert
Explore option Visitor
Unengaged
Unpersuaded
Unconverted
Tradi:onal
Conversion
Engaged
Conversion
Commi_ed
Conversion
=Convα/Vα
=Convα/Vα-­‐VU
=Convw/inpath
/Vw/inpath

53. Metric definitions: Committed conversion
• Commi3ed
conversion
=
Total
sub-­‐type
conversions
Registra:on
Total
visits
to
that
sub-­‐type s
•
Invite
a
priend
v
ideo
conversion
page(s)
•
f
• Upload
icture,
• Post
comment
or
blog
• Print
or
download
from
site

54. Conversion metrics for Home Energy Pros
100
Conversion
Rate
(%)
80
60
40
20
3.4
%
0
Tradi:onal
conver:ons
Engaged
conversions
Commi_ed
conversions
blogs
Tradi&onal
Conversion
Site
Browse Choose a
Visits
Enter
Site
& conversion
Start
Commit
to
Converted
Conversion
Convert
Explore option

55. Conversion metrics for Home Energy Pros
100
Conversion
Rate
(%)
80
60
40
20
9.4
%
3.4
%
0
Tradi:onal
conver:ons
Engaged
conversions
Commi_ed
conversions
blogs
Tradi&onal
Conversion
Engaged
Conversion
Site
Browse Choose a
Visits
Enter
Site
& conversion
Start
Commit
to
Converted
Conversion
Convert
Explore option

56. Conversion metrics for Home Energy Pros
100
Conversion
Rate
(%)
80
74.3
%
60
40
20
9.4
%
3.4
%
0
Tradi:onal
conver:ons
Engaged
conversions
Commi_ed
conversions
blogs
Tradi&onal
Conversion
Engaged
Conversion
Commi2ed
Conversion
Site
Browse Choose a
Visits
Enter
Site
& conversion
Start
Commit
to
Converted
Conversion
Convert
Explore option

57. Metrics definitions
Metrics are divided into two parts!
1. Conversion!
– Traditional conversion!
– Engaged conversion!
– Committed conversion!
2. Indices of engagement!
– Persuasion index!
• Pages are associated with conversion opportunities !
– Contribution index!
• The contribution of each page to a conversion!

58. Metric definitions: Persuasion index
• Persuasion
index
=
Commi_ed
visits
/
Visits
to
a
defined
persuasion
page
Persuasion
Index
Site
Browse Choose a
Visits
Enter
Site
& conversion
Start
Commit
to
Converted
Conversion
Convert
Explore option Visitor
Unengaged
Unpersuaded
Unconverted

59. What is a persuasion index?
A
Persuasion
Index
is
a
measure
of
the
sites
persuasive
content
• This
index
is
a
gauge
of
how
effec:vely
a
page
persuades
site
visitors
to
enter
a
conversion
path
• The
Persuasion
Index
is
a
measure
of
a
site
or
page s
persuasive
content
Persuasion
Index
Browse Start
Commit
to
&
Conversion
Convert
Explore
Unpersuaded
Unconverted
59

60. What is a persuasion index?
Persuasion
Index
Browse Start
Commit
to
&
Conversion
Convert
Explore
Unpersuaded
Unconverted
60

61. What is a persuasion index?
Persuasion
Index
Browse Start
Commit
to
&
Conversion
Convert
Explore
Unpersuaded
Unconverted

62. What is a persuasion index?
Persuasion
Index
Browse Start
Commit
to
&
Conversion
Convert
Explore
Unpersuaded
Unconverted

63. Metrics definitions
Metrics are divided into two parts!
1. Conversion!
– Traditional conversion!
– Engaged conversion!
– Committed conversion!
2. Indices of moving from engagement to conversion!
– Persuasion index!
• Pages are associated with conversion opportunities !
– Contribution index!
• The contribution of each page to a conversion!

64. What is a contribution index?
• Contribu&on
index
=
A
tally
of
the
pages
that
contribute
to
conversion

65. What is a contribution index?
• Contribu&on
index
=
A
tally
of
the
pages
that
contribute
to
conversion
1 +1 +1 +1 +1

66. What is a contribution index?
• Contribu&on
index
=
A
tally
of
the
pages
that
contribute
to
conversion
1 +1

67. What is a contribution index?
• Contribu&on
index
=
A
tally
of
the
pages
that
contribute
to
conversion
1 + 1 + 1 + 1 + 1
x2
+ 1 + 1 + 1
+ 1 + 1
Same page

68. What is a contribution index?
=1
1 +1 +1 +1 = 4
x2
Same page

69. Roadmap!
• Practices UX and AX!
• AX tools are difficult!
• Visitor life cycle!
• An example from Home Energy Pros!
• Metrics definitions!
• Persuasion in a Newsletter!
• Persuasion API – Using AX in real time!
• Summary!

70. Persuasion example: Using newsletter
Capture
Entry
Engaged
Browse
&
Persuasion
Conversion
Path
Begin
Commit
to
Converted
Page
Explore
opportunity
Conversion
Convert
Unengaged
Unpersuaded
Unconverted

71. Persuasion example: Using newsletter
Capture
Entry
Engaged
Browse
&
Persuasion
Conversion
Path
Begin
Commit
to
Converted
Page
Explore
opportunity
Conversion
Convert
Unengaged
Unpersuaded
Unconverted

72. Page
Views
0
50
100
150
200
250
2-­‐Apr
3-­‐Apr
4-­‐Apr
5-­‐Apr
6-­‐Apr
7-­‐Apr
8-­‐Apr
9-­‐Apr
10-­‐Apr
11-­‐Apr
12-­‐Apr
13-­‐Apr
14-­‐Apr
15-­‐Apr
16-­‐Apr
17-­‐Apr
18-­‐Apr
19-­‐Apr
20-­‐Apr
21-­‐Apr
22-­‐Apr
23-­‐Apr
24-­‐Apr
25-­‐Apr
26-­‐Apr
Engaged
Visits
From
Newsle3er
27-­‐Apr
28-­‐Apr
29-­‐Apr
Engagement: Linking from newsletter
30-­‐Apr
1-­‐May
5
Blog
/25
Links
4
Blogs/22
Links
3
Blogs/19
Links
2
Blogs/30
Links
1
Blog
link/2
Links

73. Engagement: Twitter
Tweets
associated
with
the
newsle_er

74. Engagement: Facebook
Facebook
posts
associated
with
the
newsle_er

75. Roadmap!
• Practices UX and AX!
• AX tools are difficult!
• Visitor life cycle!
• An example from Home Energy Pros!
• Metrics definitions!
• Persuasion in a Newsletter!
• Persuasion API – Using AX in real time!
• Summary!

76. Beyond A/B testing-Using click behaviors to tailor content!
Based on work by
Cialdini!
• Social proof!
• Scarcity!
• Authority!
A
number
of
e-­‐commerce
sites
already
use
Sales
Strategies
to
influence
consumers
buying
decision.
For
example,
they
list
the
“most
popular”
products.
We
pick
it
up
where
they
le[
it.
www.persuasionapi.com

77. Beyond A/B testing - Using click behaviors to tailor content!
Based on work by
Cialdini!
• Social proof!
• Scarcity!
• Authority!
Based on Cialdini!
• Social proof !
• Scarcity!
• Authority!
Products
can
be
promoted
using
different
Sales
Strategies:
e.g.
Bestsellers
(Social
Proof),
Almost
out
of
stock
(Scarcity),
Recommended
by
(Authority),
etc.
etc.
www.persuasionapi.com

78. Beyond A/B testing - Using click behaviors to tailor content!
Customer
A
sees
most
popular
des:na:on
Customer
B
sees
best
seller
list
most
popular
des:na:on
bestsellers
Persuasion
API
captures
which
strategy
works
best
by
INDIVIDUAL,
and
presents
future
content
using
that
informa:on.
www.persuasionapi.com

79. Roadmap!
• Practices UX and AX!
• AX tools are difficult!
• Visitor life cycle!
• An example from Home Energy Pros!
• Metrics definitions!
• Persuasion in a Newsletter!
• Persuasion API!
• Summary!

80. Practitioner takeaways!
• Define measurable success metrics early!
• Build clear and meaningful KPIs!
– Engagement!
– Persuasion !
– Committed conversion!
• Team with/make use of analytic practitioners!
• Side step the data blizzard!
• Use metrics data to !
– Refine pages within task flows!
– Identify what persuades!
#
Next generation analytics#
Improve the user journey in real time: Tailor content to individualsʼ (click)
previous preferences and behaviors!

81. !
!
! !Questions?!
!
!