A how-to guide to conducting user research field studies to understand users and their needs. Presented at UXify Bulgaria in 2015.

1. User Research in the Wild
Jim Ross
Senior UX Architect, Infragistics
jross@infragistics.com
@anotheruxguy
User Research in the Wild
Flickr: Katy Nicolson

2. User Research in the Wild
Jim Ross
Senior UX Architect, Infragistics
jross@infragistics.com
@anotheruxguy

3. z
z
What's the key to designing
an excellent user experience?
June 11, 2015 3

4. User Research in the Wild
Jim Ross
Senior UX Architect, Infragistics
jross@infragistics.com
@anotheruxguy
Understanding the users and their needs
Flickr: Highways England

5. z
z
But how do you get that understanding
of people and their needs?
June 11, 2015 5

6. Stakeholder interviews and workshops
Traditional Methods of Understanding Users
June 11, 2015 6
Flickr: Jim Larrison

7. Interviews with subject-matter experts or user representatives
Traditional Methods of Understanding Users
June 11, 2015 7
Flickr: National Assembly for Wales

8. User interviews
Traditional Methods of Understanding Users
June 11, 2015 8
Flickr: Nicholas Wang

9. Focus groups
Traditional Methods of Understanding Users
June 11, 2015 9
Flickr: Sebastiaan ter Burg

10. Surveys
Traditional Methods of Understanding Users
June 11, 2015 10

11. Usability testing
Traditional Methods of Understanding Users
June 11, 2015 11
Flickr: K2_UX

12. But what do these methods lack?
• Most of them focus on what people say, rather than what they do.
• They occur out of context.
Understanding Users and Their Needs
June 11, 2015 12

13. z
z
/ / /
So how do you get that understanding
of people and their needs?
Go out into the field to observe people in their natural context.
June 11, 2015 13

14. Understanding Users
June 11, 2015 14Flickr: Medill DC

15. Understanding Users
June 11, 2015 15Flickr: Intel Free Press

16. Understanding Users
June 11, 2015 16Flickr: State Farm

17. Field studies
• Observation and interview methods
• Solving common problems
• Overcoming objections to field studies
What We’ll Discuss
June 11, 2015 17

18. z
z
/ / /
Field Studies
What are Field Studies?
June 11, 2015 18

19. Going out “into the field” to observe and interview people in their natural context…
What are Field Studies?
June 11, 2015 19
Flickr: West Midlands Police

20. To observe firsthand their context of use:
• Users
• Tasks
• Tools and technology
• Environment
What are Field Studies?
June 11, 2015 20

21. To observe firsthand their context of use:
• Users
• Characteristics
• Knowledge
• Expectations
• Experience
• Technical skills
• Needs
What are Field Studies?
June 11, 2015 21
Flickr: UC Davis College of Engineering

22. To observe firsthand their context of use:
• Tasks
• Tasks, sub-tasks, and individual steps
• Frequency of tasks
• Workflow across people
• Goals
What are Field Studies?
June 11, 2015 22
Flickr: Anne-Sophie Leens

23. To observe firsthand their context of use:
• Tools and technology
• Technology
• Physical tools
• Documents, paperwork, manuals, etc.
• Other software and applications
What are Field Studies?
June 11, 2015 23
Flickr: Corey Seeman

24. To observe firsthand their context of use:
• Environment – physical and social
• Location: office, cubicle, outdoors, mobile
What are Field Studies?
June 11, 2015 24
Flickr: Jason Devaun

25. To observe firsthand their context of use:
• Environment – physical and social
• Location: office, cubicle, outdoors, mobile
• Activity: sitting, standing, lying, walking, etc.
What are Field Studies?
June 11, 2015 25
Flickr: Kun-chia Wu

26. To observe firsthand their context of use:
• Environment – physical and social
• Location: office, cubicle, outdoors, mobile
• Activity: sitting, standing, lying, walking, etc.
• Environmental conditions: lighting, noise, etc.
What are Field Studies?
June 11, 2015 26
Flickr: MoDOT Photos

27. To observe firsthand their context of use:
• Environment – physical and social
• Location: office, cubicle, outdoors, mobile
• Activity: sitting, standing, lying, walking, etc.
• Environmental conditions: lighting, noise, etc.
• Interaction with other people
What are Field Studies?
June 11, 2015 27
Flickr: Highways England

28. z
z
/ / /
Field Studies
Advantages of Field Studies
June 11, 2015 28

29. You see firsthand what people really do, instead of what they say they do.
Advantages of Field Studies
June 11, 2015 29
Flickr: essers

30. It’s easier for people to show you what they do, than to discuss it in the abstract.
Advantages of Field Studies
June 11, 2015 30
Flickr: MoDot Photos

31. Allows you to see all the other things involved in tasks, outside of a user interface.
Advantages of Field Studies
June 11, 2015 31Flickr: pltauk

32. Participants feel more comfortable in their own environment and act more naturally.
Advantages of Field Studies
June 11, 2015 32
Flickr: Sylvain Kalache

33. z
z
Field Study Methods
June 11, 2015 33

34. Field studies include
• Interviews
• Observation
• Contextual inquiry
Field Study Methods
Interviews Contextual inquiry Observation
June 11, 2015 34

35. Observation
Field Study Methods
June 11, 2015 35Flickr: COD Newsroom

36. Interviews
Field Study Methods
June 11, 2015 36Flickr: Dave Fayram

37. Contextual Inquiry
Field Study Methods
June 11, 2015 37Flickr: UGA College of Ag & Environmental Sciences - OCCS

38. You can combine these methods.
Field Study Methods
June 11, 2015 38
Flickr: Highways England

39. z
z
How to Perform a Field Study
June 11, 2015 39

40. Steps in the Process
• Planning
• Conducting the Study
• Analyzing the Data
• Creating Deliverables
How to Perform a Field Study
June 11, 2015 40

41. z
z
/ / /
How to Perform a Field Study
Planning
June 11, 2015 41

42. Conduct user research at the beginning or before a project begins.
Planning
June 11, 2015 42

43. Plan what you want to learn from the research.
• Why are you conducting the research?
• What is the goal?
• What do you want to learn?
Planning
June 11, 2015 43

44. Review any existing information about the users and their domain.
Planning
June 11, 2015 44
Flickr: Terry FreedmanFlickr: Jim Larrison

45. Choose the techniques you’ll use to gather that information.
Planning
June 11, 2015 45
Interviews Contextual Inquiry Observation

46. Determine how many participants to include by considering:
• The number of user groups
• The number of tasks you need to observe
• The need for repetition
Planning
June 11, 2015 46
5 5 5 5

47. Determine how many participants to include by considering:
• The limitations of time, budget, and travel
Planning
June 11, 2015 47
Flickr: 401(K) 2013

48. Recruit and schedule Participants.
Planning
June 11, 2015 48

49. Create a discussion guide.
Planning
June 11, 2015 49

50. Create a discussion guide.
• Introduction
• Questions
• Tasks to observe
Planning
June 11, 2015 50

51. Create a discussion guide.
• Keep it as a loose guide.
Planning
June 11, 2015 51

52. Plan how you will capture the data.
• Handwritten notes
Planning
June 11, 2015 52
Flickr: Geek Calendar

53. Plan how you will capture the data.
• Typing notes in a laptop or tablet
Planning
June 11, 2015 53
Flickr: ilouque

54. Plan how you will capture the data
• Recording video
Planning
June 11, 2015 54
Flickr: Gunnar Bothner-By

55. Plan how you will capture the data
• Recording video
Planning
June 11, 2015 55

56. Plan how you will capture the data
• Recording audio
Planning
June 11, 2015 56

57. Plan how you will capture the data
• Livescribe Smartpen
Planning
June 11, 2015 57

58. Decide whether you will allow additional observers.
• Two is the ideal number of people to conduct field studies.
Planning
June 11, 2015 58

59. If you have additional observers, give them rules:
• You’re the leader
• Refrain from talking, making noises, or sending signals with body language
• Where to sit
• When to ask questions
• How to ask neutral, non-leading questions
Planning
June 11, 2015 59

60. Prepare the participants for your visit.
• Individual session
• Not an interview
• Observing tasks
• In your usual location
• Save work to show during the session
Planning
June 11, 2015 60

61. z
z
/ / /
How to Perform a Field Study
Contextual Inquiry
June 11, 2015 61

62. Contextual Inquiry
June 11, 2015 62Flickr: Terry Whalebone

63. General format
• Greeting
• Introduction
• Initial interview
• Observing tasks
• Follow-up questions
Contextual Inquiry
June 11, 2015 63

64. Greeting
• Make the participant comfortable
• Be informal
• Establish a good rapport
Contextual Inquiry
June 11, 2015 64

65. Greeting
Contextual Inquiry
June 11, 2015 65

66. Greeting
Get people to open up and talk to you (Oprah):
• Make the participant comfortable
• Establish trust and rapport
• Listen and let the participant talk most of the time
• Be curious
• Be interested
Contextual Inquiry
June 11, 2015 66

67. Greeting
Be neutral (Freud):
• Avoid biasing or leading the participants.
• Get the participant talking.
• Listen.
• Don’t judge.
Contextual Inquiry
June 11, 2015 67

68. Introduction
• Purpose of the research
• Your philosophy – you represent the user
• Explain what you’ll be doing
• Get permission to record
Contextual Inquiry
June 11, 2015 68

69. Initial interview
• Background information
Contextual Inquiry
June 11, 2015 69

70. Initial interview
• Background information
• Their experience and involvement in the subject
you’re researching
• What kind of car are you interested in?
• Where are you at in the car buying process?
• What kind of research have you done?
• How do you determine what type of car you’ll buy?
Contextual Inquiry
June 11, 2015 70

71. Initial interview
• Background information
• Their experience and involvement in the subject
you’re researching
• Discuss their tasks
Contextual Inquiry
June 11, 2015 71

72. Transition from the interview to contextual inquiry…
Contextual Inquiry
June 11, 2015 72
Flickr: J J

73. Transition from the interview to contextual inquiry
Conducting the Study
June 11, 2015 73Flickr: Rhys A.

74. Transition from the interview to contextual inquiry
• Requires the participant to change their mindset from:
Contextual Inquiry
June 11, 2015 74
Interview
Passive, answering questions
Task Demonstration
Actively teaching

75. Transition from the interview to contextual inquiry
• Make it clear that you’re going to do something
very different.
Contextual Inquiry
June 11, 2015 75

76. Transition from the interview to contextual inquiry
• Tell the participant:
• Actually perform the task.
• Go through the entire task, step-by-step.
• Tell me what you’re doing.
• You’re the master, and I’m the apprentice.
Contextual Inquiry
June 11, 2015 76
Flickr: Esther Dyson

77. At the start of each task
• Find out what led up to this point.
• What steps came before?
• Who else was involved?
• How does the task get to them?
Contextual Inquiry
June 11, 2015 77
Flickr: Fabrizio Salvetti

78. The participant performs the task.
• Goes through each step in the task
• Describes what he/she is doing
Contextual Inquiry
June 11, 2015 78
Flickr: Southern Arkansas University

79. Flickr: Southern Arkansas University
Ask probing questions as needed.
• “Why?”
• “Tell me more about that.”
• “What happens next?”
• “Can you show me how that works?”
Contextual Inquiry
June 11, 2015 79Hall, Erika. "Interviewing Humans." A List Apart The Full. N.p., 10 Sept. 2013. Web. 21 May 2015.
Flickr: Southern Arkansas University

80. During the tasks, pay attention to:
• Actions and explanations
• Tools and technology used
• Information, paperwork, documents
• People they interact with
• The environment
Contextual Inquiry
June 11, 2015 80
Flickr: For Inspiration Only

81. Facilitating a contextual inquiry can be very difficult.
Contextual Inquiry
June 11, 2015 81Flickr: Sasquatch I

82. Facilitating a contextual inquiry can be very difficult.
Contextual Inquiry
June 11, 2015 82
Flickr: Sasquatch I

83. Facilitating the session involves:
• Observing
• Listening
Contextual Inquiry
June 11, 2015 83Flickr: You Belong In Longmont

84. Facilitating the session involves:
• Assessing whether you understand
• Asking questions
• Should I ask a question about this?
• How should I phrase it to avoid bias?
• When should I ask the question?
• Noting questions to ask later
Contextual Inquiry
June 11, 2015 84
Flickr: Ethan Lofton

85. Facilitating the session involves:
• Assessing the answer
• Did I understand the answer?
• How should I rephrase it?
Contextual Inquiry
June 11, 2015 85
Flickr: Ethan Lofton

86. Facilitating the session involves:
• Ensuring the participant keeps showing and
explaining the task
Contextual Inquiry
June 11, 2015 86
Flickr: Ethan Lofton

87. Facilitating the session involves:
• Assessing how well the session is going
• Are you getting the information you need?
• Is this really the task you want to see?
• Is the participant staying on track?
• Do you need to lead the participant in another
direction?
• Are there other tasks that you want to see next?
Contextual Inquiry
June 11, 2015 87
Flickr: Anders Sandberg

88. Facilitating the session involves:
• Keeping track of the time
• How much time is left in the session?
• Is this task taking too long?
• Will we still have time to see the other tasks?
• Maintaining a positive rapport with the participant
• Taking notes
Contextual Inquiry
June 11, 2015 88Flickr: AaltoFablab

89. How can you handle the competing demands of facilitation?
• Include repetition in participants and tasks
• Don’t take notes during the session
• Review recordings and take notes from those
• Delegate tasks to observers
• Learn and improve with experience
Contextual Inquiry
June 11, 2015 89

90. Follow-Up Questions
After each task and at the end of the session
• What happens next?
• Who performs the next step?
• Reconfirm your understanding
• Ask any remaining questions
Contextual Inquiry
June 11, 2015 90
Flickr: You Belong In Longmont

91. Closing
• Thank the participant
• Take photos of the environment,
tools, documents, technology, etc.
Contextual Inquiry
June 11, 2015 91
Flickr: rallenhill
Flickr: Mike Beltzner

92. z
z
/ / /
How to Perform a Field Study
Analyzing the Data
June 11, 2015 92

93. Type up your notes between sessions.
• Helps you assess what you’ve learned so far
• Avoids too much to type up at the end
Analyzing the Data
June 11, 2015 93
Flickr: Rachel Johnson

94. Analysis methods: spreadsheet
Analyzing the Data
June 11, 2015 94

95. Analysis methods: affinity diagram
Analyzing the Data
June 11, 2015 95

96. z
z
/ / /
Yeah, But What If…?
Solutions to Common Problems
June 11, 2015 96

97. The participants have tasks that can’t be interrupted.
Solutions to Common Problems
June 11, 2015 97
Flickr: World Bank Photo Collection

98. The participants have tasks that can’t be interrupted.
Solutions to Common Problems
June 11, 2015 98
Flickr: yooperann

99. The participants have tasks that can’t be interrupted
Solutions
• Interview before
• Observe the tasks without interruption
• Interview after
Solutions to Common Problems
June 11, 2015 99Flickr: Andy G

100. People want to do the session in a conference room.
Solutions to Common Problems
June 11, 2015 100
Flickr: Chris Miuccio

101. People want to do the session in a conference room
Solutions
• Explain the importance of observing tasks in their usual context.
• If that doesn’t work, cancel the session.
Solutions to Common Problems
June 11, 2015 101

102. Participants try to change it to a group session.
Solutions to Common Problems
June 11, 2015 102
Flickr: Pat Kight

103. Participants try to change it to a group session
Solutions
• Emphasize ahead of time that these are individual sessions.
• Offer to schedule the additional people for their own sessions.
• Be flexible to adapt when it does make sense to include a second participant.
Solutions to Common Problems
June 11, 2015 103

104. Participants try to change it to a group session
Solutions
• Be flexible to adapt when it does make
sense to include a second participant.
Solutions to Common Problems
June 11, 2015 104
Flickr: IAEA Imagebank

105. The participant keeps slipping into interview mode.
Solutions to Common Problems
June 11, 2015 105
Flickr: You Belong In Longmont

106. The participant keeps slipping into interview mode.
Solutions
• Make a dramatic transition from the interview to the observation of tasks.
• Keep reminding that this is not an interview, you want to observe tasks in detail.
• Instead of asking, “How do you…?” Say, “Show me how you….”
Solutions to Common Problems
June 11, 2015 106

107. It becomes a complaint session.
Solutions to Common Problems
June 11, 2015 107
Flickr: Penumbra

108. It becomes a complaint session.
Solutions
• Clarify that you’re there to observe and understand their tasks.
• Ask them to save their list of problems or improvements for the end of the session.
Solutions to Common Problems
June 11, 2015 108

109. The participant can’t show you a task.
Solutions to Common Problems
June 11, 2015 109Flickr: Penumbra

110. The participant can’t show you a task.
Solutions - ask the participant to:
• Meet when the participant normally performs that task
• Save work to show you during the session
• Walk you through what they normally do, as best they can
Solutions to Common Problems
June 11, 2015 110

111. The subject matter is very complicated and unfamiliar.
Solutions to Common Problems
June 11, 2015 111Flickr: Faculty of Medicine NTNU

112. The subject matter is very complicated and unfamiliar.
Solutions
• Learn as much as you can before you conduct your sessions.
• Interview stakeholders first, to get an overview.
• Observe multiple participants performing the same tasks.
• Review the recordings.
Solutions to Common Problems
June 11, 2015 112

113. The participant talks above your level of understanding.
Solutions to Common Problems
June 11, 2015 113Flickr: NASA HQ Photo

114. The participant talks above your level of understanding.
Solutions
• Remind them that they’re the expert and you’re the apprentice.
• Ask dumb questions to remind them how much you don’t know.
Solutions to Common Problems
June 11, 2015 114

115. z
z
Objections to User Research
June 11, 2015 115

116. “It takes too long. We don’t have the time.”
Objections to User Research
June 11, 2015 116

117. Flickr: TaxRebate.org.uk
“It costs too much. We can’t afford it.”
• Time for the researcher and designer
• Recruiting
• Incentives
• Travel
Objections to User Research
June 11, 2015 117

118. “We can’t travel to the participants.”
Objections to User Research
June 11, 2015 118Flickr: Brian Fagan

119. z
z
Overcoming Objections to User Research
June 11, 2015 119

120. Ensure that projects are scoped with user research as an activity.
Overcoming Objections to User Research
June 11, 2015 120
Rosenfeld Media: Leah Buley

121. Perform user research outside of projects.
Overcoming Objections to User Research
June 11, 2015 121
Sy, Desiree. "Adapting Usability Investigations for Agile User-centered Design." Journal of Usability Studies 2.3 (2007): n. pag. UPA. UPA, May 2007. Web. 6 June 2014.

122. Start with a small study to show the value of user research.
Overcoming Objections to User Research
June 11, 2015 122

123. Find participants in your local area to limit travel.
Overcoming Objections to User Research
June 11, 2015 123

124. Divide the research between several people in different locations, each conducting the
research with local participants.
Overcoming Objections to User Research
June 11, 2015 124

125. Limit the number of participants by focusing only on the primary user group.
Overcoming Objections to User Research
June 11, 2015 125
Doctors Nurses Hospital Administrators Technicians

126. Narrow the scope of your research and shorten the sessions.
• Fewer questions you want to answer
• Fewer tasks to observe
Overcoming Objections to User Research
June 11, 2015 126
Flickr: Juhan Sonin

127. Shorten the analysis and deliverables
• Don’t type up your notes
• Produce only high-level, informal deliverables
Overcoming Objections to User Research
June 11, 2015 127
Flickr: chicgeekuk

128. Conduct remote contextual inquiries to reach people that you can’t travel to.
Overcoming Objections to User Research
June 11, 2015 128Flickr: HeatherLWilliams

129. z
z
/ / /
The key to designing
an excellent user experience
Understanding the users and their needs
June 11, 2015 129

130. June 11, 2015 130Flickr: Fly For Fun
Flickr: U.S. Army RDECOM
Flickr: Leonardo RizziFlickr: Va DOT
Flickr: wistechcolleges

131. June 11, 2015 131
Thank You

132. z
z
Resources for More Information
June 11, 2015 132

133. Recruiting participants
• Recruiting Better Research Participants: Jim Ross
• Recruiting User Research Participants by Email: Jim Ross
User research methods
• Observing the User Experience: A Practitioner's Guide to User Research: Elizabeth
Goodman, Mike Kuniavsky, Andrea Moed
Resources for More Information
June 11, 2015 133

134. Interviewing
• Interviewing Users: How to Uncover Compelling Insights: Steve Portigal
• Interviewing Humans: Erika Hall
Resources for More Information
June 11, 2015 134

135. Contextual inquiry
• Contextual Design: Hugh Beyer, Karen Holtzblatt
• Rapid Contextual Design: Karen Holtzblatt, Jessamyn Burns Wendell, Shelley Wood
• Why are Contextual Inquiries So Difficult?: Jim Ross
• Remote Contextual Inquiry: A Technique to Improve Enterprise Software: Lynn
Rampoldi-Hnilo
Resources for More Information
June 11, 2015 135

136. Recording user research data
• Capturing User Research: Jim Ross
Handling observers in user research sessions
• Observing User Research: Jim Ross
User Research Deliverables
• Communicating User Research Findings: Jim Ross
Resources for More Information
June 11, 2015 136

137. Getting buy-in to conduct user research
• It’s Our Research: Getting Stakeholder Buy-In for User Experience Research Projects:
Tomer Sharon
Resources for More Information
June 11, 2015 137

138. June 11, 2015 138
Thank You