JD Buckley: "Communicating the ROI of UX from The Enterprise to The Streets"
Enterprise UX 2018 • June 14-15, 2018 • San Francisco, CA, USA
http://enterpriseux.net
18. THE ENTERPRISE
ATTITUDE METRICS
TOP TASKS
COMPANY KPI’S
+
NPS
SALES GROWTH
CALL CENTER VOLUME
PROFITS
TASKS & ATTITUDE
METRICS
UX METRICS
BASELINE
TASKS & ATTITUDE
METRICS
REDESIGN
+
19. THE ENTERPRISE
Task 2Task 1 Task 3 Task 4 Task 5
Top Tasks Benchmark Research
What Are Users Most Important Tasks? How Did Users Perform
When Attempting to Accomplish Tasks?
Data visualization by Kristy Lee
22. THE ENTERPRISE
1. Clarify representative audience
2. Identify customer tasks
3. Define UX metrics
4. Benchmark the current experience
5. Compare redesign to the benchmark
6. Connect to company KPIs
28. THE ENTERPRISE
UX METRICS:
ATTITUDE & TASK METRICS
COMPANY METRICSASSUMPTIONS: BENCHMARK VS. REDESIGN
Net Promoter
SUS
NPS score will increase
System Usability Score (SUS) will increase
Efficiency will increase resulting in improved time on taskEfficiency • Increase in Sales Growth
Effectiveness Completion rate will increase resulting in increased effectiveness
Perceived Difficulty Perceived difficulty will decrease resulting in improved ease of use
Perceived Time Perceived time will improve resulting in improved ease of use
Satisfaction Usability will improve resulting in increased user satisfaction • Increase in Customer Experience NPS
Trust/Reliability will increase
Trust/Reliability
About the Product • Increase in Client Retention
Trust/Reliability about
Product Information
Trust/Reliability about Product Information will increase
• Decrease in Contact Volumes
• Decrease in Contact Volumes
• Decrease in Contact Volumes
• Decrease in Contact Volumes
• Increase in Customer Experience NPS
• Increase in Customer Experience NPS
30. THE ENTERPRISE
T O P TA S K # 1
“ I N E E D T O … ” S TAT E M E N T D E S C R I B I N G T O P TA S K S I N U S E R S O W N L A N G U A G E ”
A R C H E T Y P E
# 1
P R I M A RY
A R C H E T Y P E
# 2
S E C O N D A RY
T O P TA S K # 2
“ I N E E D T O … ” S TAT E M E N T D E S C R I B I N G T O P TA S K S I N U S E R S O W N L A N G U A G E ”
Roadmap Item #1 Roadmap Item #2
User Goal
Problem SolvedProblem Solved
Success MetricsSuccess Metrics
Roadmap Item #3 Roadmap Item #4 Roadmap Item #5
A R C H E T Y P E
# 2
P R I M A RY
Roadmap Item #3 Roadmap Item #4 Roadmap Item #5
User Goal User Goal User Goal
Problem Solved Problem Solved Problem Solved
Connecting Top Tasks to Road Map Items
32. THE ENTERPRISE
1
2
3
4
5
Employment Tax Wage Garnishments
3.47
4.03
3.923.85
3.71
3.36
Benchmark Study 1 Study 2
Satisfaction
Not at all satisfied
Very satisfied
Neutral
Product 1
Benchmark Study 1 Study 2Benchmark Study 1 Study 2
Product 2
39. I N S P I R E D B Y G R E AT
I N N O VAT O R S
THE STREET
1. Clarify representative audience
2. Identify customer tasks
3. Define UX metrics
4. Benchmark the current experience
5. Compare redesign to the benchmark
6. Connect to company KPIs
40. I N S P I R E D B Y G R E AT
I N N O VAT O R S
THE STREET
1. Clarify representative audience
2. Identify customer tasks
3. Define UX metrics
4. Benchmark the current experience 5. Compare redesign to the benchmark
6. Connect to company KPIs&
&
43. I N S P I R E D B Y G R E AT
I N N O VAT O R S
THE STREET
44. THE STREET
Advocacy
National Health
Education Standards
Decision-Making
Interpersonal CommunicationSelf-Management
Goal-Setting Assessing Information
Core Concepts Analyzing Influences
Step 2 - Identify Customer Tasks
45. I N S P I R E D B Y G R E AT
I N N O VAT O R S
one
THE STREET
U X M E T R I C S A S S U M P T I O N S : A N A L O G U E V S . D I G I TA L “ W H E R E ’ S D A RY L ” K P I ’ S
Engagement
Increase in Engagement will lead to more teacher downloads of
kits and purchase of class materials
• Use a decision-making process to examine risky social and dating situations.
Health Standards - S27
Awareness
Increase in number of students who improve scores on post program
survey vs. pre program survey
Increase in views of short “Where’s Daryl” videos
• Increase awareness of how the presence of weapons increases the risk of serious
violent injuries
• Increase awareness of behavioral and environmental factors associated with major
causes of death in the US.
• Analyze how the media portrays guns
Health Standards - S3, S9, S16, S17,
Usefulness
Increase in teachers implementing program by jurisdiction
Increase in number of students who have completed course
Increase in number of middle schools who have in each jurisdiction
who have completed courses
• Apply a decision-making process to a variety of situations that impact mental,
emotional, and social health
• Analyze the importance of setting personal boundaries for privacy, safety, and
expressions of emotions and opinions.
Health Standards - M20, M24
Education
Increase in % of students who improve scores on post program
survey vs. pre
• Increase in students ability to identify and report dangerous gun related situations
• Demonstrate skills to avoid or escape a potentially violent situation
Health Standards - S3 & S9, S20, M30
Knowledge Sharing
Increase in teachers sharing knowledge of program across schools,
districts, jurisdiction
Increase in social media mentions/sharing , (Snapchat, Instagram) by
students/teachers
• Support changes to promote safety in the home, school, and community.
Health Standards - S37
Steps 3 & 6 - Define UX Metrics & Connect to KPIs
46. THE STREETKordic, Timothy & Plant, Aaron & Ruffino, Elisa & Amatullo, Mariana & H. Moon, Maria & Lamb, Kerry. (2014).
“Where's Daryl?”: Results of a gun violence prevention-education program in Los Angeles Unified School District.
Results & Outcomes
725 students completed the baseline survey
712 students completed the follow-up
After students participated in the program:
•Students had a statistically significant increase in 5 of 16
items assessing negative attitudes towards guns
•Students had a significant increase in knowledge in 4 of
7 items
•Overall gain in knowledge was significant
Teachers were very positive about “Where’s Daryl?”
•86% said it was engaging for students
•71% said lessons were effective
•Only 43% of teachers had discussed guns with students
before implementing the program
Step 4 & 5 - Benchmark Current Experience & Compare New Experience to Benchmark
47. I N S P I R E D B Y G R E AT
I N N O VAT O R S
THE STREET
“Where’s Daryl” | DesignMatters