Christian Rohrer: "​Insight Types That Influence Enterprise Decision Makers" Enterprise UX 2015 • May 13, 2015 • San Antonio, TX, USA http://enterpriseux.net

1. .
Intel Confidential
Christian P. Rohrer, PhD | Chief Design Officer, Intel Security, Consumer Products
​ Insight at Scale:
​ Insight Types That Influence
Enterprise Decision Makers
Presented at UX Enterprise Conference, San Antonio, May, 2015

2. .
Intel Confidential
Topics and Overview
1. Context
• Me: My background and perspective
• You: Internal or External to the “Enterprise”
• Them: Customers, Administrators, End Users
2. UX Model and Research Methods
• User Needs are Central to UX
• User Research Methods
• Research Methods Across Product Development
3. Insight Challenges and Opportunities
• Access to Insights
• Insights that Drive Decision Making
• The Golden Trapezoid

3. .
Intel Confidential
​ Context

4. .
Intel Confidential
4
1987 19991992 2004
Chief Design Officer
VP Experience Design
Consumer,
Mobile Enterprise
Senior Director of User
Experience Design
Director of UER
and Product Research
Director of User
Experience Research
Unix/Xenix Tech Support
Engineer & Trainer
UC Santa Cruz
Computer Science
(Honors BA)
Stanford University
Cognitive Science,
Education, HCI (PhD)
1998
Network Computer, Inc.
(aka Liberate)
The Santa Cruz
Operation (aka Caldera)
2011
A bit about me and my perspective
2015
Ethnographic
Field Researcher
2008
*
* Intel acquired McAfee, March, 2011.
How many of you knew?
*

5. .
Intel Confidential
Intel Security
Experience Design Team
≈ 50 Amazing people | 6 offices around the world
5

6. .
Intel Confidential
“Security”
Great User
Experiences

7. .
Intel Confidential
Enterprise
Employee
Enterprise
Employee
Now let’s talk about YOU…
Are you an Internal or External Enterprise UX team?
Enterprise
Employee
Enterprise
Employee
Enterprise
Employee
Enterprise
Employee
Enterprise
Employee
Enterprise
Employee
An “Enterprise” (company)
2. A B2B
business that
builds products
for external
“enterprise”
customers?
Design
User
Research
Product
Mgmt
1. An IT or internal
department, building for
internal customers?
Design
User
Research
IT
Develop-
ment

8. .
Intel Confidential
THEM: these Enterprise Employees, who are “THEY?”
8
VP/Director
Tech Ops
CISO
CIO
Security/
Systems
Architect
Network
Ops
Manager
Desktop
Ops
Manager
Employee
End User
Enterprise
Employee
They could be many types of people – for example:

9. .
Intel Confidential
In the world of enterprise security, THEY might be
classified into something like this:
9
Decision Makers
TechnicalSophistication
VP/Director
Tech Ops
CISO
CIO
Security/
Systems
Architect
Network
Ops
Manager
Desktop
Ops
Manager
Employee
End User
Employee
End User
“Users”
“Choosers”

10. .
Intel Confidential
Customer
Customer
Or more simply: { End Users | Admins | Customers }
Customer
Customer
Admin
Admin
End User
End User
An “Enterprise” (company)

11. .
Intel Confidential
11
What do “They” need to do to succeed?
Potential examples from an Enterprise context:
Buy
Deploy
Control
Report
Use
Not Get
Fired
Customer
Admin
End User

12. .
Intel Confidential
12
​ UX Model and Research Methods

13. .
Intel Confidential
13
See also, Jesse James Garrett’s The Elements of User Experience
Christian Rohrer (2006-2015)
User
Needs
A Simple Model of User Experience
ms
secs
mins
Insights
on these
layers?

14. .
Intel Confidential
User Research Methods
• Many methods available
• Many pros & cons
• But what to use and when?
• Methods on a 3D landscape:
o Qualitative vs. Quantitative
o Behavioral vs. Attitudinal
o Context of Product Use
14
?
?
?

15. .
Intel Confidential
15

16. .
Intel Confidential
16

17. .
Intel Confidential
17

18. .
Intel Confidential
18

19. .
Intel Confidential
19

20. .
Intel Confidential
20
• Natural: Examine natural behavior and attitudes
• Scripted: Focus insight topics or enforce consistency
• De-contextualized: Issues under study are broader than product usage
• Hybrid: Creatively use limited form of product use to meet research goals
Context of Use

21. .
Intel Confidential
21
Usability Lab
Studies
• Single-user using
product/prototype
• Lab-based
• Usually scripted
• Behavior > Attitude

22. .
Intel Confidential
22
Ethnographic
Field Studies
• A technique
inspired by the
field method used
by sociocultural
anthropologists
• Observation of
work or natural use
of products
• Goal: understand
through the eyes
of the observed
• Of all qualitative
methods, the most
powerful/flexible

23. .
Intel Confidential
23
Focus Groups
& Interviews
• Group or one-on-
one discussions
about specific
topics (brand,
marketing, or
products/services)
• By definition, only
talking – not using
product/prototype

24. .
Intel Confidential
24
Clickstream
Analysis & A/B
Testing
• Analysis of data
stored in logs (web
or SW telemetry)
on what users click
• A/B Tests: give a
random sample of
users alternative
version of website
design; compare
logs of behavior to
current site design
Image source: TechCrunch

25. .
Intel Confidential
25
Surveys
• Asking large
numbers of users
what they think in
a structured way
• Email surveys:
invite participants
via email
• Intercept surveys:
randomly invite a
percentage of
users on a site or
using an app for
their opinion about
the site/app

26. .
Intel Confidential
26
• Do we have the right idea?
• If not, what is better?
• Who is the target?
• What makes it work?
• Can be Qual or Quant
Concept Testing

27. .
Intel Confidential
27

28. .
Intel Confidential
28
Google “user
research methods”
for more info on
the landscape:

29. .
Intel Confidential
29
​ Insight Challenges and Opportunities

30. Soooooo many potential sources of information….
30
Decision Makers
TechnicalSophistication
VP/Director
Tech Ops
CISO
CIO
Security/
Systems
Architect
Network
Ops
Manager
Desktop
Ops
Manager
Employee
End User
Employee
End User
Enterprise Users and Customers

31. Security/
Systems
Architect
Let’s start with an Internal UX team, developing solutions
inside of an organization
31
Decision Makers
TechnicalSophistication
VP/Director
Tech Ops
CISO
CIO
Network
Ops
Manager
Desktop
Ops
Manager
Employee
End User
Employee
End User
An “Enterprise” (company)
The Good News:
• Easy access to
users and
customers
and
• No competition!
(Wait, is this
good?)
1. An IT department with
a UX team, building for
internal customers
UX/
Design
Prog
Mgmt
IT
Develop-
ment

32. The challenges for an Internal UX team, developing solutions
inside of an organization
32
Decision Makers
TechnicalSophistication
VP/Director
Tech Ops
CISO
CIO
Security/
Systems
Architect
Network
Ops
Manager
Desktop
Ops
Manager
Employee
End User
Employee
End User
An “Enterprise” (company)
The Bad News
Challenge:
• No competition
therefore
• No pressure to
deliver a high
quality UX
1. An IT department with
a UX team, building for
internal customers
UX/
Design
Prog
Mgmt
IT
Develop-
ment

33. The challenges for an Internal UX team, developing solutions
inside of an organization
33
Decision Makers
TechnicalSophistication
VP/Director
Tech Ops
CISO
CIO
Security/
Systems
Architect
Network
Ops
Manager
Desktop
Ops
Manager
Employee
End User
Employee
End User
An “Enterprise” (company)
Challenge:
• Internal
customers:
rarely held to
“good UX” as a
measure of
success
1. An IT department with
a UX team, building for
internal customers
UX/
Design
Prog
Mgmt
IT
Develop-
ment
Internal
Customer
“Good UX?
Meh.”

34. The challenges for an Internal UX team, developing solutions
inside of an organization
34
Decision Makers
TechnicalSophistication
VP/Director
Tech Ops
CISO
CIO
Security/
Systems
Architect
Network
Ops
Manager
Desktop
Ops
Manager
Employee
End User
Employee
End User
An “Enterprise” (company)
Challenge:
• No time to do
User Centered
Design
1. An IT department with
a UX team, building for
internal customers
UX/
Design
Prog
Mgmt
IT
Develop-
ment

35. The challenges for an External UX team, developing solutions
for an external enterprise customer
35
Decision Makers
TechnicalSophistication
VP/Director
Tech Ops
CISO
CIO
Security/
Systems
Architect
Network
Ops
Manager
Desktop
Ops
Manager
Employee
End User
Employee
End User
Enterprise Users and Customers
2. A business
that builds
products for
external
“enterprise”
customers
UX/
Design
Develop-
ment
Product
Mgmt

36. Challenge: There may be a specific department (e.g., Sales)
that owns the relationship with the enterprise customer…
36
Decision Makers
TechnicalSophistication
VP/Director
Tech Ops
CISO
CIO
Security/
Systems
Architect
Network
Ops
Manager
Desktop
Ops
Manager
Employee
End User
Employee
End User
Enterprise Users and Customers
2. A business
that builds
products for
external
“enterprise”
customers
UX/
Design
Develop-
ment
Product
Mgmt

37. Challenge: You may be blocked from accessing the enterprise
customers and their users if the benefits of doing so aren’t clear
37
Decision Makers
TechnicalSophistication
VP/Director
Tech Ops
CISO
CIO
Security/
Systems
Architect
Network
Ops
Manager
Desktop
Ops
Manager
Employee
End User
Employee
End User
Enterprise Users and Customers
2. A business
that builds
products for
external
“enterprise”
customers
UX/
Design
Develop-
ment
Product
Mgmt

38. Challenge: If you DO manage to get access to enterprise
customer information, it will often be about BUYING DECISIONS
38
Decision Makers
TechnicalSophistication
VP/Director
Tech Ops
CISO
CIO
Security/
Systems
Architect
Network
Ops
Manager
Desktop
Ops
Manager
Employee
End User
Employee
End User
Business Users and Customers
2. An business
that builds
products for
external
enterprise
customers
UX/
Design
Develop
-ment
Product
Mgmt
What to BUY

39. How are BUYING DECISIONS typically made?
39
Decision Makers
TechnicalSophistication
VP/Director
Tech Ops
CISO
CIO
Security/
Systems
Architect
Network
Ops
Manager
Desktop
Ops
Manager
Employee
End User
Employee
End User
Business Users and Customers
2. An business
that builds
products for
external
enterprise
customers
UX/
Design
Develop
-ment
Product
Mgmt
CUSTOMERS (deciding to buy)
• How much does it cost?
• Does it “check all the
boxes” I’m supposed to
check?
• Will it get me fired if it
doesn’t work?

40. “What to Buy” leads to “What to Build”
40
Decision Makers
TechnicalSophistication
VP/Director
Tech Ops
CISO
CIO
Security/
Systems
Architect
Network
Ops
Manager
Desktop
Ops
Manager
Employee
End User
Employee
End User
Business Users and Customers
2. An business
that builds
products for
external
enterprise
customers
UX/
Design
Develop
-ment
Product
Mgmt
Decision: What to BUILD
What to BUY

41. Challenge2:
How good is this “data”?
Will this serve all in the ecosystem?
41
Decision Makers
TechnicalSophistication
VP/Director
Tech Ops
CISO
CIO
Security/
Systems
Architect
Network
Ops
Manager
Desktop
Ops
Manager
Employee
End User
Employee
End User
Business Users and Customers
An business
that builds
products for
external
enterprise
customers
UX/
Design
Develop
-ment
Product
Mgmt
Anecdotal
Third-hand
Self-Reported
Conflicting

42. As with Consumer, there are many types of enterprise “Users”
42
Decision Makers
TechnicalSophistication
VP/Director
Tech Ops
CISO
CIO
Security/
Systems
Architect
Network
Ops
Manager
Desktop
Ops
Manager
Employee
End User
Employee
End User
Business Users and Customers
2. An business
that builds
products for
external
enterprise
customers
UX/
Design
Develop
-ment
Product
Mgmt
USERS

43. End Users: “If I have to use it, can it at least be USABLE?”
Better yet: USEFUL
43
Decision Makers
TechnicalSophistication
VP/Director
Tech Ops
CISO
CIO
Security/
Systems
Architect
Network
Ops
Manager
Desktop
Ops
Manager
Employee
End User
Employee
End User
Business Users and Customers
2. An business
that builds
products for
external
enterprise
customers
UX/
Design
Develop
-ment
Product
Mgmt
“Make
the
product
USABLE”

44. Admin users: “I have so little time - don’t make me think. Fit
into my workflow. Most importantly, don’t let me screw up.”
44
Decision Makers
TechnicalSophistication
VP/Director
Tech Ops
CISO
CIO
Security/
Systems
Architect
Network
Ops
Manager
Desktop
Ops
Manager
Employee
End User
Employee
End User
Business Users and Customers
An business
that builds
products for
external
enterprise
customers
UX/
Design
Develop
-ment
Product
Mgmt
“Make it EASY
TO DEPLOY,
MAINTAIN, and
USEFUL in my
workflow”

45. Challenge: Insights from different sources from different
perspectives to different decision makers = CONFLICT
45
VP/Director
Tech Ops
CISO
CIO
Security/
Systems
Architect
Network
Ops
Manager
Desktop
Ops
Manager
Employee
End User
Employee
End User
Business Users and Customers
An business
that builds
products for
external
enterprise
customers
UX/
Design
Develop
-ment
Product
Mgmt
Insights
on:
product
USABILITY
Insights on: what
makes these
decision-makers
BUY
Inevitable
Conflict !
Insights on:
DEPLOY,
MAINTAIN, and
workflow USE

46. Customer
Customer
It’s a complex ecosystem… Q: How do we make sense of it?
Internal
Customer
Customer
Admin
Admin
End User
End User
B2B
UX/
Design
Develop-
ment
Product
Mgmt
Internal/IT
UX/
Design
Prog
Mgmt
IT
Develop-
ment
SALES

47. ​ A: In the context of TIME…

48. .
Intel Confidential
Understand
Conceive
Design
Develop
Launch
Strategize
Optimize
Assess
Key Q:
How
well did
we do?
Key Q:
How shall we
do it?
Key
Q:
What shall we do?
&
Why
?
Qualitative & Quantitative
§ Ethnographic Field Studies
§ Interviews/Focus Groups
§ Clickstream Analysis
§ Survey results (NPS)
§ Concept Testing
Mostly Qualitative
§ Participatory design
§ Concept Studies
§ Prototype Evaluations
§ Usability (lab) studies
§ Desirability studies
Mostly Quantitative
§ A/B Testing
§ Usability benchmarks
§ Online UX Assessments
§ Surveys (CSAT, NPS)
§ Clickstream Analysis

49. .
Intel Confidential
Strategize
Key
Q:
What shall we do?
&
Why
User
Needs

50. .
Intel Confidential
Strategize
Key
Q:
What shall we do?
&
Why
Qualitative & Quantitative
§ Ethnographic Field Studies
§ Interviews/Focus Groups
§ Clickstream Analysis
§ Survey results (NPS)
§ Concept Testing
User
Needs
Customer
End User
UX/
Design
Product
Mgmt
Admin

51. .
Intel Confidential
Optimize
Key Q:
How shall we
do it?
User
Needs

52. .
Intel Confidential
Optimize
Key Q:
How shall we
do it?
Mostly Qualitative
§ Participatory design
§ Concept Studies
§ Prototype Evaluations
§ Usability (lab) studies
§ Desirability studies
End User
UX/
Design
Admin

53. .
Intel Confidential
Assess
Key Q:
How
well did
we do?
? User
Needs

54. .
Intel Confidential
Assess
Key Q:
How
well did
we do?
?
Mostly Quantitative
§ A/B Testing
§ Usability benchmarks
§ Online UX Assessments
§ Surveys (CSAT, NPS)
§ Clickstream Analysis
User
Needs
Customer
End User
UX/
Design
Develop-
ment
Product
Mgmt
Admin

55. .
Intel Confidential
​ You Can’t Do It All

56. .
Intel Confidential
56

57. .
Intel Confidential
57

58. .
Intel Confidential
58
​ Thanks!
@christianrohrer