Presented at Web Unleashed on September 16-17, 2015 in Toronto, Canada
More info at www.fitc.ca/webu
Measuring UX
with Hira Javed
OVERVIEW
User experience is the sum of a series of interaction. It’s about motivations, expectations, behaviours and attitudes, and that’s hard to quantify. In order to know whether something is working, a signal or an indication of progress is needed, and progress cannot be measured without a clear sense of what success looks like. Metrics are the signals that show whether a UX strategy is working or not. But signals can be ambiguous.
This presentation will focus on going beyond basic metrics for evaluating the impact of UX changes, and identifying effective KPIs for measuring the quality of UX.
OBJECTIVE
To understand the limitations of basic traffic metrics when it comes to measuring UX, and discuss Google’s HEART Framework for a holistic measure of quality UX.
TARGET AUDIENCE
Anyone involved in strategizing and creating user experiences.
FIVE THINGS AUDIENCE MEMBERS WILL LEARN
Challenges with measuring UX
Limitations of basic traffic metrics
Characteristics of effective UX KPIs
Google’s HEART Framework for measuring the quality of UX
Applying the HEART Framework
5. “User experience represents the
perception left in someone’s mind
following a series of interactions
between people, devices, and events –
or any combination thereof.”
- Eric Reiss
9. In order to know whether something is
working, you need a signal.
Metrics are the signals that show whether your
UX strategy is working.
10. But…signals can be ambiguous.
Easily collected, yet not always important.
Measurable, yet not always informative.
11. Basic traffic metrics are easy to track and give
a good baseline on how your site is doing, but
they often aren’t very useful for evaluating the
impact of ux changes.
12. Effective UX KPIs are:
Behaviour based
Indicator of performance
Key to the business
Unique to the business
Easy to measure
Diagnostic
Source: http://www.uie.com/articles/power_of_ux_kpi/
13. Behaviour-based: Behavioural information is
a critical indicator of user’s experience.
Performance indicators: A good KPI predicts
an important change in the business,
hopefully with enough of a lead time to
react if necessary.
14. Key to the business: Five business goals KPIs
can be related to:
1) increasing revenues
2) decreasing costs
3) increasing market share
4) increasing revenue from existing
customers and
5) increasing shareholder value
15. Unique to the business: Generic KPIs
produce generic results. KPIs should be
specific to the how the business runs.
Easy to measure: KPIs are measured
frequently, therefore they should be easy to
track.
Diagnostic: KPIs should help identify what
the problem is.
17. The HEART Framework
Developed by Google’s research team.
Measures the quality of user experience.
Can be applied to a specific feature or the
whole product.
26. Goals Signals Metrics
Happiness Users share story page
with their social
networks
An increase in traffic
from referral sites
Number of visitors
from referral sites
Engagement Users spend more time
on our website
The amount of
content they
consume
Number of stories
they read per
session
Adoption Users subscribe to CBC
newsletters
Users visit the
subscription page
Number of new
subscribers
Retention Users check CBC news
app frequently during
the day
Users read the story
whenever a
notification is sent
out
Number of
pageviews on a story
linked to a news
alert
Task Success Users can comment on
a story
Users complain
about the
commenting process
Number of
complaints about
commenting process
27. Lessons Learned
• This is a lot harder than it looks!
• Not all elements of the HEART Framework will
apply to you. Need to customize.
• Bridges the gap between product creators
and product users. Creates empathy.
• Helps make direct connection between
product KPI and UX strategy.