“It takes too long." "We don’t have the budget." "We don’t really need it, we can just optimise once we’ve gone live.” Sound familiar? As UX embraces agile as a project delivery approach, research seems get left out in the cold. Rather than shivering and complaining about it though maybe we just need to stick two fingers up to these assumptions and dare to do things a little differently! In her workshop at UX People, Nina helped delegates explore how to bring research in from the cold on agile projects. There were tips on getting the research basics right (effective participant recruitment and facilitation techniques), and delegates were given the opportunity to road-test their facilitation and analysis skills in an agile-friendly framework (full exercises not available in this presentation). If you're looking to arm yourself with some practical skills, and a research approach that will blow those assumptions about speed, cost and the lack of value out of the water then this workshop would have been for you, but you'll have to make do with this SlideShare presentation instead!

1. A G I L E - F R I E N D LY U S E R R E S E A R C H
UX PEOPLE 2013
Nina Belk
Lead Research Strategist
DigitasLBi
Twitter: @ninabelk
LinkedIn: uk.linkedin.com/in/ninabelk/

2. DISCLAIMER
This is a workshop about research,
not about agile project delivery.

3. Objections:
“It costs too much.”
“It takes too long.”
“We don‟t need it. We‟re building
something new, people will just ask
for a faster horse!”

4. SOLUTION:
“We can just optimise based
on analytics data once
we‟ve launched.”

5. A cautionary tale…

6. Sorry, I can‟t show you slides to illustrate the or tell you who
the client is.
Here‟s the summary: when we did some post launch user
research and discovered the client assumptions were badly
wrong, and then redesigned some key pages and tools on
the site, conversion went through the roof.

7. The Agile Manifesto (for software development)
“We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer [client] collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on
the right, we value the items on the left more.”
Source: http://agilemanifesto.org/

8. The 12 Principles of Agile (software development)
1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
2. Welcome changing requirements, even late in development. Agile processes harness change for the customer's
competitive advantage.
3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the
shorter timescale.
4. Business people and developers must work together daily throughout the project.
5. Build projects around motivated individuals. Give them the environment and support they need, and trust them
to get the job done.
6. The most efficient and effective method of conveying information to and within a development team is face-toface conversation.
7. Working software is the primary measure of progress. Agile processes promote sustainable development.
8. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
9. Continuous attention to technical excellence and good design enhances agility.
10. Simplicity--the art of maximizing the amount of work not done--is essential.
11. The best architectures, requirements, and designs emerge from self-organizing teams.
12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior
accordingly.
Source: http://agilemanifesto.org/

9. My Brief for an Agile-Friendly Approach:
– Fast
– Collaborative
– Minimal documentation

10. The Traditional Approach to User Experience Research
!
Discussion
Guide
Discussion
Guide
Client
activities
Your
activities
Brief
Discussion
Guide
Discussion
Guide
Participants recruited
Discussion Guide Document
User Research Sessions
Against an agreed
recruitment brief.
You draft a discussion guide, the client
reviews it and provides feedback. You
then make amendments and then
send it back to the client for approval.
Participants will be given scenarios that encourage
a natural exploration of the key pages we‟re
researching.
Client and design team observe.

11. The Traditional Approach to User Experience Research
Detailed
Report
Report
Presentation:
Findings and
recommendations
presented &
discussed.
Client
activities
Your
activities
Detailed
Report
Data analysis
Report Prepared
Report Presentation
You analyse the data for patterns.
You put together a lengthy PowerPoint
deck and it‟s sent to the client and
design teams
You present your findings and recommendations
to the client and discuss the implications of these
with them and the design team before agreeing on
actions.

12. The Agile-Friendly Approach
Collaborative
Note Taking
Client & Design Team
activities
Your
activities
Discussion
Guide Workshop:
Observers note their
observations on post-it
notes and stick to wall.
Findings
Workshop:
Review notes
captured during
research sessions
& agree actions.
Agree areas to
explore in the
research.
Brief
Findings
Summary
Discussion
Guide
Participants recruited
Discussion Guide Document
User Research Sessions
Outputs
Against an agreed
recruitment brief.
Workshop to agree discussion
guide topics and structure, the
output of which we will document
for reference not approval
Participants will be given scenarios that
encourage a natural exploration of the key
pages we‟re researching.
Following the workshop we will put
together a short summary of the
agreed actions for reference not
approval.

13. Recruitment
Don‟t give the recruitment criteria
away in the screener questions!
Try to word questions in an open
manner

14. TASK ONE:
Fix the recruitment screener to
prevent professional research
participants getting into your
study.
TIP TO REMEMBER:
Avoid giving the criteria away in the question.

15. Discussion Guide Workshops

16. The Discussion Guide Workshop
Discussion Guide Workshop Agenda:
1.
Explain the purpose of the workshop
2.
Explain how you will run the research sessions:
–
–
–
–
There‟s no script because it feels unnatural
You start with contextual questions to help you build a picture of participants and warm them up to the content of the
session
You then look at what you want to evaluate
Finally you wrap-up with some overall evaluation of what the participants have seen during the course of the session.
3.
With the designers, walkthrough the journeys they would like you to evaluate – these would ideally be printed and
stuck on the wall.
4.
Brainstorm the research questions you have specific to the experience together and write these on post-it notes
and stick alongside the relevant parts of the journey.
5.
Create realistic scenarios that allow the participants to get into the right frame of mind and ensure the research
questions will be addressed.
6.
Brainstorm contextual research questions that can be asked at the start of the research sessions
7.
Review the standard wrap-up questions and amend where needed:
–
–
–
–
Overall what did you think of what we‟ve looked at today?
Was there anything you particularly liked that you want to highlight?
Is there anything you would change or add to improve it for you?
Any final comments?
FINALLY: Document what you agree (no client review).

17. TASK TWO:
Create your discussion guide
POINTS TO REMEMMBER:
•
Walkthrough the journeys you want to evaluate
•
Together, brainstorm the research questions you have specific to the experience and write these on post-it notes.
Where they are screen specific stick the post-it alongside, where they‟re not create a group of questions that
apply to all screens.
•
Write a realistic scenario(s) that allows the participants to get into the right frame of mind and ensure the research
questions will be addressed.
•
Brainstorm contextual research questions that can be asked at the start of the research sessions.
•
Review the standard wrap-up questions and amend where needed.
•
Document on the template provided.

18. Facilitating

19. Research labs can be
intimidating places
for participants.

20. Help them relax by
making them feel
at home.
Image sourced from: http://www.flickr.com/photos/kellysue

21. Introducing Participants to the Session
– An introduction to you
– An overview of what the session is about
– Explain the tech set-up
– Outline the structure
– What you‟d like them to do.

22. Open with your scenario
“I‟d like you to
imagine that…”
Facilitation
“What do you think
will
happen next?”
“What‟s going on
here?”
“What‟s this page
about?”
“What did you find
difficult about
that?”
Asking „why‟ a lot can feel
like an interrogation
“What can you do
here?”
Encourage them to think aloud
It‟s hard not to ask leading
questions, but don‟t panic…
“Would you find
that useful?”
“Can you give me
an example of
when?”
Top Tip
Repeat the last thing
they said in a rising
intonation.
…you can always
qualify them.

23. Collaborative Analysis

24. Image sourced from: Detroit Publishing Company photo via Library of Congress website.
“Can I have a
faster horse please
Mr. Ford?”
Watch, listen and interpret. Don‟t just report!
It‟s seriously
slow getting
from A to B on
this horse and
cart.

25. Setting up the Observation Room and the Observers
1. Have all pages printed and
stuck on brown paper
2. Have an area dedicated to
‘key observations’
3. Give observers a guide to
effective note taking

26. Guide For Observers
1. Watch what they do
2. Listen to what they say, and how they say it
3. Compare what you see and what you hear
If they struggled but said it was easy trust their actions not
their words.
4. Try and interpret what‟s going on rather than just reporting.
Goal – get a “because” on every post-it note
5. Mark-up your post-it notes with the participant number and the
screen they were on (post-it rain is a pain).
6. Discuss observations together after each session
7. Pull out the three most important issues to address.
Summarise these on a post-it not for each participant and
place on the key observations sheet.

27. TASK THREE:
Run a short research session
TIPS FOR FACILITATORS:
•
Put your participant at ease
•
Open questions and then explore interesting avenues
TIPS FOR OBSERVERS:
•
Watch what they do
•
Listen to what they say and how.
•
Compare what they do with what they say
•
Observation + because (try and interpret)

28. Findings Workshops

29. Findings Workshop Preparation
Summarise key
findings for each
screen
Group
observations

30. The Findings Workshop
Agenda:
• Recap on the key research questions
• Nominate someone as the actions scribe
• Walk through the experience with the discussion guide as
your agenda
• Highlight the key findings and discuss the actions the
team could take to address these.
• Agree the priority of the design changes
• Review the personas and discuss if these need to be
updated based on anything you‟ve learned about during
the research.

31. TASK FOUR:
Prepare and run a findings workshop
POINTS TO REMEMBER
•
Group similar observations
•
Summarise the key findings
•
Use your discussion guide as a workshop agenda
•
Discuss findings and agree actions

32. Objections:
“It costs too much.”
“It takes too long.”
“We don‟t need it. We‟re building
something new, people will just ask
for a faster horse!”

33. – It can cost you more not to do
research (remember my cautionary
tale?)
– If it‟s collaborative with minimal
documentation, it‟s faster and
cheaper.
– The real skill of a researcher is
interpretation not facilitation!

34. Thank You