Presented at the UXPA September event on User-Centred Design and Agile

1. eewei@emoti.vu / @emotivu / Emoti.vu
Metrics Driven Design
Better define success, deliver impact, make money
Eewei Chen
@Ultraman
http://www.ml.sun.ac.za/wp-content/uploads/2009/08/social_graph.png
“Efforts and courage are not enough without purpose
and direction.”
John F. Kennedy

2. 20 years

3. What is success?

4. Stories, workflows, wireframes, interactions
My Flights
ALERT:
Manila to Cebu
Book A Flight
Get The Best Exchange Rates
Flight Status
CP100 - OK
Check-in
Available (1)
Destination Guides CEB Club
Commercial Partners About Cebu Pacific
Sign-in
Register
Shake Iphone to view more Special Flight Offers
MANILA TO
BORACAY
One way from
P780
DISRUPTION: Manila to Cebu delayed
Abstract Concrete
UX Assets?

5. Working software
Burn down
Collaboration
Refactoring
Agile
good enough

6. Disruption
no longer exist
no one can live without
Survival?

7. Can UX success be measured?

8. Pack selection
+ 213%
Checkout completion
+ 252%
User satisfaction*
+ 237%
Sky Upgrades
Free Trial uptake
+ 137%
Subscriptions uptake
+ 187%
User satisfaction*
+ 223%
Now TV Subscriptions
Query resolution
+ 120%
Info retrieval speed
+ 239%
User satisfaction*
+ 174%
Sky.com Nav
Problem:
Poor trial and subs uptake
Solution:
• New marketing animation
• More effective brand messages
• Timed marketing triggers
• Remote usability test (2K users)
• 4 week POC, 12 weeks
Problem:
Too many choices, low upgrade
conversion
Solution:
• Usage based recommendations
• 9 packs down to 3
• Quick check out
• MVT Testing to 1% traffic
• 3 week POC, 8 weeks delivery
Problem:
Inaccurate FAQs and MySKY tasks
Solution:
• Super nav drop down
• Quick tasks
• Simplified IA
• Remote usability test (2K users)
• 2 week POC, 6 weeks delivery
*In App customer survey feedback form

9. BUT what if there isn’t much to go on?
(Case study)

10. Digital Airline Experiences are fragmented
Fear & Punishment works??
Apple wants it all
Many platforms

11. Stay connected with travellers
• Data: Passenger info
• Need: More connected mobile digital experience
• ROI: Mobile Ad wallet share
• UX: Useful traveller engagement throughout lifecycle
Airline
Airline
Airline
Advertiser
Advertiser
Advertiser

12. Opportunity
Mobile Advertising
US$11.4 Billion(i)
Mobile Ad spend in 2013
£500 million
UK spend in 2012 is most per user (ii)
(i)
-­‐
h'p://www.gartner.com/newsroom/id/2306215
(ii)
-­‐
h'p://www.guardian.co.uk/media/2012/oct/09/mobile-­‐adverDsing-­‐500-­‐million-­‐pounds-­‐2012
“Connecting
with Travellers”
Trend
Gap
Channel
Vision

13. Data (any)
Insight
Business Traveller
going on a Business Trip
buys a ticket 7 Days or
less before travel
52/41 = (127) index

14. Business Traveller
• 23% of all travellers, Frequent flyer (175)
• male (123), 50 Years Old (139), Company Chairman (233)
• Earns 90K / month (133)
• Android smartphone / tablet user (NA)
• Unaware of Airline Loyalty Program (109)
Behaviours(unique insight)
• Books travel & hotel < 7 days in advance (127)
• Pays for tickets at airline ticket office (115), airport (117) airline
call centre (111) and travel agent (115).
• Checks-in at airline desk (106)
• Buys advertised goods at airport (128), in-flight (113), later (117).
Spends over 250 (191) on perfume, electronics, watches,
sunglasses, toys, DVDs/Music
• Research In-Flight magazine (114) and travel sites (132).
• Plans next trip going to airport (129), in airport lounge (129) and
on the flight (111)
• Uses laptop on the plane (124)
• Doesn’t value airline membership (67% of business)
Personality
• Last minute business trip planner
• Appreciates quality
• Will use something if it makes his life easier
• Wants respect and a personalised service.
• Prefers human face-to-face contact
• Impulse buyer, influenced by advertising
• Likes to stay informed and up-to-date
• Wants to maximise work time
Barriers
• Does not trust social media recommendations
• Does have time to do much research
• Money not an issue but wants to know it is well spent
Output, Outcome, Impact
• Deal alerts<7 days before travel to increase Deal
Section Visits by 25%& Ad CPM by 25%
• Improve mobile usage (095) and mobile boarding pass value
(101) to reduce non-web and non-mobile bookings (all of above)
• Faster mobile check-in (092) to reduce physical in-person
check-ins (by all of above) and check-in time by 25%.
• Product and service recommendations (200): business (1.35),
food (1.06), hotel (1.03) and consumer products(1.04),
entertainment (1.05) to improve Ancillary CPM by 200%
• Personalised Ads: Automotive (1.31), Corporate (1.29),
Conference (1.19), Electronics (1.10), Tourism (1.01) and in-flight
retail choices (0.81) to increase online/mob CPM and conv (TBD).
• Travel updates (0.94) and info to increase recommend to
friend (092) and repeat booking purchase (101)
• In-flight wifi to increase laptop usage (124)
•(numbers)
represent
indexed
data
analyDcs
Deal Alerts<7 Days
in advance of travel
Data

15. Empathy
Books travel & hotel <7 Days in
advance (127)
Deal Alerts<7 Days
in advance of travel
Increase Ancillary CPM
rate by 25%
Does not have time to
do research
Increase PVs by 25%
Last minute planner
OUTPUT
OUTCOME
IMPACT
Barriers
Personality
Behaviour
(numbers)
represent
indexed
GPS
data

16. Success
Output
Software we build. The materials we produce.
Outcome
The immediate change due to delivering output.
Impact
The larger change we see over time
Outcome
Impact
Output

17. Assumptions
We believe by Output,
we will Outcome*
leading to Impact*
We believe by creating deal alerts,
we will increase visits to deals section by 25%
leading to an increase in Ad CPM by 25%
*QualitaDve
or
quanDtaDve
measurement
Outcome
Impact
Output

18. What do we do first?

19. Value (relative)
EASY
HIGH VALUE
DIFFICULT
Quick wins Differentiator
Parity / building blocks Bin
LOW VALUE
Deal Alerts
Home screen
advert
Basic geo-location
*if
we
had
to
build
the
capability
ourselves.
Therefore...
Get
client
to
build
and
enable
Mobile CMS
In-flight wifi*
iOS Push Alerts
ROI:
Customer value x Personas
Business value A
Business value B
Ease of implementation
Dependency level

20. Wireframes
Deal Alerts <7 Days in
advance of travel
My Flights
ALERT:
Manila to Cebu
Book A Flight
ALERT / ADVERT
Flight Status
CP100 - OK
Check-in
Available (1)
Destination Guides CEB Club
Commercial Partners About Cebu Pacific
Sign-in
Register
Shake Iphone to view more Special Flight Offers
MANILA TO
BORACAY
One way from
P780
DISRUPTION: Manila to Cebu delayed
Flight notifications
Ancillary service notifications
OFF
Connect with Facebook
Disconnect Twitter
Connect with LinkedIn
Settings
Share this app
Contact us
Edit your personal details
CEB Club profile
Get in touch
Social Networks
Push notifications
ON
CEBU PACIFIC:
You've just arrived. Book a taxi
and save 15% today
CEBU PACIFIC:
Discounted fares to Boracay this
week starting from PHP 780
SPECIAL OFFERS
Discount Airtime Tariffs
SPECIAL OFFERS
ALERT / ADVERT
MANILA TO
BORACAY
One way from
P780
.....
Terms & Conditions
Additional info on Special offers
BOOK NOW
Done
Outcome
Set explicit notifications
(time, location, price etc.)
Receive well timed push
notification
See on page alerts on
key pages* (e.g. HP)
Click to view target page
(Destination Page + AD)
*Assume
persistent
signed-­‐in
state
Impact
Output
Output
Output

21. What more can we measure?

22. Passenger
info
from
Airline
Origina'on
Des'na'on
Gender
Departure
date
Return
date
Language
Cabin
class
Age
Airline
loyalty
level
Ticket
type
(open
or
return)
Na'onality
Implicit
Click
path
analysis
Page
view
/
Preview
Time
on
page
Shared
on
FB
and
TwiHer
Email
Geo-­‐locate
/
Check-­‐in
Time
of
day
Implicit
but
heavy
AI,
Machine
Learning*
Sentence
analysis
Mood
analysis
Other
behavioural
paHern
analysis
Explicit
Wizards
Ques'onnaires
Booking
Search
Save
SeSngs
/
Preferences
Likes
Good Start Differentiation
Track it

23. What about making money?

24. Success can be Everywhere*
Triggers*
Holistic
AARRR!**
*BJ
Fogg,
Behavioral
Analysis:
h'p://www.behaviormodel.org/
**
Dave
McClure:
Pirate
Metrics:
h'p://www.slideshare.net/dmc500hats/startup-­‐metrics-­‐for-­‐pirates-­‐long-­‐version

25. Don’t forget to experiment

26. *In App customer survey feedback form
Validation
• Guess
• Consequence
• Experiment
Cornell U, 1964
• Assumption
• Validate
• MVP
September 2011
Richard Fenyman Eric Ries
If it disagrees with
experiment, it’s WRONG!

27. In Conclusion
• Problem / Goal
• Opportunity
• Data
• Success= Outputs, Outcomes, Impact
• Assumptions
• Empathy
• Value Map
• Wireframes
• Track & Capture
• Make Money
• Experiment

28. Metrics Driven Design
How to define success and deliver more impact
Eewei Chen
@Ultraman
http://101DesignIngredients.com