1.
Collaborative
Research
with Erika Hall (@mulegirl)
UXLX

2.
Hello!

3.
I have a
question...

4.
Do you enjoy being
right?

5.
You are correct!

6.
YESSS!

7.
p0wned!

8.
No.

9.
?

10.
>? !

11.
Dogma

12.
Flickr/Chris Voll

13.
Ego!

14.
Why?

15.
People!

16.
Collaboration!

17.
Text
Agenda

18.
Text
Agenda
9–10:30
10:30–11
11–12:30
Introduction.
The research process
Questions and activities
Understanding your organization
Break
User research
Analysis
Models and reports
Finale

19.
Barriers

20.
Overcoming Objections

21.
Laziness

22.
Laziness
Fear

23.
Laziness
Fear
Lunch

24.
Laziness
Fear
Lunch
Following

25.
Laziness
Fear
Lunch
Following
Losing Control

26.
Laziness
Fear
Lunch
Following
Losing Control
Sharing

27.
Shared
Understanding

28.
We don’t have
the time.

29.
We don’t have
the money.

30.
We don’t have the
expertise.

31.
We’re already A/B
testing

32.
Everyone wants better products,
faster.

33.
No one wants to read a report.

34.
What is your
experience?

35.
Research +
Collaboration

36.
A design project is
a series of decisions.

37.
Data doesn’t
change
minds.

38.
What
is
What ought
to be

39.
Design-Led
Research-Led
Expert
Mindset
Participatory
Mindset
Users seen as subjects
Users seen as partners
Design-led
with
expert mindset
Design-led
with
participatory mindset
Research-led
with expert mindset
Research-led
with participatory
mindset
Dubberly Design Office

40.
Goal Driven
Skeptical Mindset
Increase chance of success
Reduce risk
Willing to question the value of any approach

41.
Team + Goal + Reality = Good

42.
One Simple
Process

43.
Form
Questions
Analyze
Data
Gather
Data

44.
Form
Questions
Analyze
Data
Think
Critically

45.
Form
Questions
Analyze
Data
Observe

46.
Form
Questions
Analyze
Data
Interview

47.
Form
Questions
Analyze
Data
Read

48.
Form
Questions
Analyze
Data
Read
Experiment
Interview
Observe
Think

49.
Personal
View
Personal
View
Personal
View

50.
Shared
Reality

51.
Research
is a Craft

52.
Form
Questions
Analyze
Data
Gather
Data

53.
Questions determine results.

54.
Questions give research meaning.

55.
Research high-priority questions.

56.
Good Questions
Specific
Actionable
Practical

57.
A Bad Question
“What do people
think about pets?”

58.
A Better Question
“How do single
urban adults
choose and
acquire a pet?”

59.
A Bad Question
“What do people
do around here all
day?”

60.
A Better Question
“How do editors
and designers work
together?”

61.
The Best Question
The unknown with
the most risk.

62.
Bias

63.
Bias: Something that causes an
influence or prejudice

64.
Confirmation Bias:
You selectively weight the
information that confirms
what you already believe.

65.
Sampling Bias:
Your sample of research
subjects isn’t sufficiently
representative.

66.
Interviewer Bias:
You insert your opinion into
interviews.

67.
Social Desirability Bias
People don’t say the true
things that they worry will
make them look bad.

68.
Ease
Clear Display
Related
Primed Idea
Good Mood
Feels True
Feels Familiar
Feels Good
Feels Effortless
Daniel Kahneman

69.
Feeling confident?
It’s not a good sign.

70.
You might have a bad case of
Dunning-Kruger.

71.
Critical
Thinking

72.
Critical Thinking
Disciplined
Self-correcting
Clear
Logical

73.
Uncritical Thinking
“I hate yellow, so a
yellow website
won’t succeed.”

74.
Critical Thinking
“I hate yellow, but
based on the
evidence, it might
work for our
audience.”

75.
Critical Thinking
“I don’t know.”

76.
Activities!

77.
Form
Questions
Analyze
Data
Gather
Data

78.
Questions
About
Users
ProductOrg
Competition
InterviewsInterviews
Usability
Testing
A/B
Testing
Contextual
Inquiry
Literature
Review
SWOT
Analysis
Brand
Audit
Usability
Testing
Competitive
Analysis
Heuristic
Analysis
Descriptive
Evaluative
Evaluative
Evaluative
Analytic
Analytic
Generative
Descriptive

79.
Research
Activity
Topic
Purpose
Time
Money

80.
Phone Interviews
What do we
need to know
about?
What kind of
decision will it
inform?
How long do we
have?
What is our
budget?
Contextual Inquiry
In-Person Interviews
Usability Testing
Competitive
Analysis

81.
Why not just
make a
prototype?

82.
20
If we only test bottle openers, we
may never realize customers prefer
screw-top bottles.
– Victor Lombardi, Why We Fail

83.
Topics

84.
Organizational
Research

85.
Stakeholders

86.
Stakeholders

87.
Executives

88.
Sales People

89.
Customer Service

90.
Editors

91.
Production Team

92.
Organizational research
helps you with:
Requirements
Politics
Workflow
Capabilities
Goodwill

93.
Requirements
What are the top
business priorities
for this project/
product?

94.
Politics
What does
success mean to
the individual
stakeholders?

95.
Workflow
Do we have to
change how
people are
working together
to be successful?

96.
Workflow
How do we have
to change how
people are
working together
to be successful?

97.
Workflow
How can we
possibly change
how people are
working together?

98.
Capabilities
What are the
strengths and
weaknesses of our
team?

99.
Capabilities
Where is the
internal expertise?

100.
Goodwill
How can this
project make your
job easier (or
harder)?

101.
Get them alone

102.
Basic Stakeholder Questions
What is your title? How long have you been in this role?
What are your essential duties and responsibilities?
What does a typical day look like?
Who are the people you work most closely with? How is that going?
What does success mean from your perspective, what will have changed
for the better once this project is complete?
Do you have any concerns about this project?
What do you think the greatest challenges to success are? Internal and
external?

103.
For each stakeholder, note the following:
What’s their general attitude toward this project?
What’s the goal as they describe it?
To what extent are this person’s incentives aligned with the project’s
success?
How much and what type of influence do they have?
Who else do they communicate with on a regular basis?
To what extent does this stakeholder need to participate
throughout the project, and in which role?
Is what you heard in harmony or in conflict with what you’ve
heard from others throughout the organization?

104.
Stakeholder power moves
“Why are you asking me this?”
“I don’t understand that question. It doesn’t make any sense.”
“I don’t feel comfortable talking to you about that.”
“No one pays attention to anything I have to say, so I don’t know why I
should bother talking to you.”
“How much more time is this going to take?”

105.
Practice!

106.
10 minutes practice. Find a partner. Take turns.
What is your title? How long have you had this job?
What are your essential duties and responsibilities?
What is a typical day like?
Who are the people you work most closely with? How is that going?
What do you think the greatest challenges to your success are?
Internal and external?

107.
Empathy

108.
Break!

109.
To Review
Team + Goal + Shared Reality = Good
Research is a simple process you can apply to however you work. You
shouldn’t be dogmatic.
Even though this sounds obvious, some people will resist this because
questions can feel threatening.
Facts will not change the minds of people who are threatened.
You need to appeal to what you know is important to them, and fit
your facts into their story.
So, understanding what is important to your stakeholders is necessary
for design and research to succeed.

110.
User
Research

111.
Photo: Flickr/theloushe
Ethnography

112.
How to do bad user research:
Ask people what they want.

113.
How to do bad user research:
Ask people what they like.

114.
Never ask users what
they want or like.

115.
The Four Ds of Design
Ethnography

116.
Deep Dive
Daily Life
Data Analysis
Drama

117.
“...true ethnography reveals not just
what people say they do, but what they
actually do.”
–PARC

118.
Photo: Flickr/lintmachine

119.
The Art of
The Interview

120.
Interviewing is not talking.

121.
Interviewing is listening.

122.
Good Interviewers:
Know Your Question
Warm Up
Shut Up

123.
Interview Structure:
Introduction
Body
Conclusion

124.
Introduction:
Smile
Express gratitude
Describe the process
Ask to record
Warm up questions

125.
Body:
Ask open-ended questions
Probe for more
Allow silence
Use questions as checklist

126.
Conclusion:
Transition to wrap-up
Ask if there is anything else
Thank for time

127.
You are the host
You are the student

128.
Out of your comfort zone,
and into theirs.

129.
Interview Checklist
Create a welcoming atmosphere to make participants feel at ease.
Always listen more than you speak.
Take responsibility to accurately convey the thoughts and
behaviors of the people you are studying.
Start each interview with a general description of the goal,
but be careful of focusing responses too narrowly.
Avoid leading questions and closed yes/no questions. Ask
follow-up questions.
Prepare an outline of your interview questions in advance, but don’t be
afraid to stray from it.
Also note the exact phrases and vocabulary that participants use.

130.
Look for
Goals
Priorities
Tasks
Motivators
Barriers
Habits
Relationships
Tools
Environment

131.
Roles
Interviewer
Notetaker
Observer

132.
Practice!

133.
Interview Scenario
You work for an e-commerce site
that wants to develop a new service
to help people give gifts.
The goal of the research is to
identify unmet needs people might
have with regard to giving gifts.

134.
Interview Practice
Break into groups of 3-4 people
1 interviewee, 1 interviewer , 1 notetaker, 1
observer (optional),
Switch in 15 minutes
2 rounds

135.
Look for
Goals
Priorities
Tasks
Motivators
Barriers
Habits
Relationships
Tools
Environment

136.
How did that go?

137.
How about a
focus group?

138.
14
“Even when the subjects are well
selected, focus groups are supposed to
be merely the source of ideas that need
to be researched.”
–Robert K. Merton, Sociologist,
invented focus groups

139.
Everybody Lies

140.
Competitive
Research

141.
How else might your
target customer solve
the same problem?

142.
Competitive Review
What do they say they offer?
Who is their customer? How is this the same or different from your
target audience or users?
What are the key differentiators—the factors that make them uniquely
valuable to their target market, if any?
How do the user needs or wants they’re serving overlap or differ from
those that you’re serving or desire to serve?
What do you notice that they’re doing particularly well or badly?
Based on this assessment, where do you see emerging or established
conventions in how they do things, opportunities to offer something
clearly superior, or good practices you’ll need to adopt or take into
consideration to compete with them?

143.
Your target customers
have to love you
more than they
hate change.

144.
(Usability)
Testing

145.
A good research activity:
•Answers a key question
•Addresses identified
assumptions
•Informs specific decisions
•Involves your team
•Fits your level of expertise
•Fits your schedule and budget

146.
•Fundamentally research is a simple process
•There are many activities and definitions
•No pressure!
•Select the methods that inform decisions
•Begin by understanding your organization
•Never ask what people like
•People are lazy, forgetful creatures of habit
•Keep each other honest
•Practice and learn

147.
Research and Collaboration
Working together across disciplines and making decisions based on
evidence shouldn’t be hard, but they can be.
Done right, research and working collaboratively reinforce each other
through a shared understanding of reality.
Start with your goal in mind, not with any process or buzzword.
Asking questions and cutting across traditional roles can both be
threatening to the established order.
Commit to clear communication and critical thinking.
Research questions follow from goals, assumptions, and risk.
Always have a framework and a plan.

148.
Creating Meaning
From Data

149.
1. Compile data
2. Analyze
3. Identify Insights
4. Create Model

150.
Basic Analysis
Closely review the notes.
Look for interesting behaviors, emotions, actions, and verbatim quotes.
Write what you observed on a sticky note (coded to the
source, the actual user, so you can trace it back).
Group the notes.
Watch the patterns emerge.
Rearrange the notes as you continue to assess the patterns.

151.
Observation
Observation
Observation
Observation
Observation
Observation

152.
Observation
Observation
Observation
Observation
Observation
Observation
Observation

153.
Collaborates
on purchases
Observation
Observation
Observation
Observation
Observation
Observation
Observation

154.
Collaborates
on purchases
Uses several
devices
Observation
Observation
Observation
Observation
Observation
Observation
Observation

155.
Collaborates
on purchases
Uses several
devices
Needs
affirmation
Observation
Observation
Observation
Observation
Observation
Observation
Observation

156.
Ground rules
Acknowledge that the goal of this exercise is to better understand the
context and needs of the user. Focus solely on that goal.
Respect the structure of the session. Refrain from identifying larger
patterns before you’ve gone through the data.
Clearly differentiate observations from interpretations (what happened
versus what it means).
No specific solutions until after you’ve gone through insights and
principles. Solutions come next.

157.
Practice!

158.
20 minutes analysis.
Break into groups of 6-8 people
Each group work together to fill out one diagram with the strongest
patterns.
Negotiate and advocate for your perspective.

159.
Look for
Goals
Priorities
Tasks
Motivators
Barriers
Habits
Relationships
Tools
Environment

160.
20 minutes analysis.
Break into groups of 6-8 people
Each group work together to fill out one diagram with the strongest
patterns.
Negotiate and advocate for your perspective.

161.
How did that go?

162.
Models

163.
Reporting

164.
You are collaborating with
your future selves.

165.
Research ReportStudy Title
Date Completed
Research Goal
Activities
Related Decisions
Key Insights
Supporting Observations
Recommended Actions

166.
A useful report supports
Clear goals
Shared values
Access to information
Clear decision-making

167.
You decide if it’s important for the report to be
Informing?
Inspiring?
Focusing?
Remembering?
Recording?
Deciding?

168.
Finale

169.
In summary
Research creates a shared understanding of reality.
Asking questions is uncomfortable. Embrace that feeling.
A truly collaborative approach and environment is necessary for research to be
effective, and it also makes it more fun.
Clear goals and good questions are required.
Choose only the research activities that answer real questions and inform your top
priority design and development decisions.
Practice! Observe and listen every day.
Document! Report! Share! It’s easy to lose what you learn.

170.
Any questions?

171.
Additional sources:
Designing Together by Dan A. Brown
http://www.designingtogetherbook.com/
LeanUX by Jeff Gothelf
http://www.leanuxbook.com/
Remote Research by Nate Bolt & Tony Tony Tulathimutte
http://rosenfeldmedia.com/books/remote-research/
Interviewing Users by Steve Portigal
http://rosenfeldmedia.com/books/interviewing-users/
Google Ventures Library | Design
http://www.gv.com/library/design/
Pacific Standard Magazine
http://www.psmag.com/
Helsinki Design Lab (closed, but excellent publications still available)
http://www.helsinkidesignlab.org/pages/publications

172.
Brief books for people who make websites No.
9
JUSTENOUGH
RESEARCH
Erika Hall
You might
enjoy the
book.
www.abookapart.com
For 15% off, use code: UXLXJER14