Communicating user research findings effectively so that people can understand them, believe them, and know how to act on the recommendations can be challenging. You may feel that you’ve delivered a successful presentation, but later you find that the recommendations aren’t acted upon. Ideally, our clients are as interested in our user research findings and recommendations as we are and find them valuable, but without the proper understanding, clients can express a variety of negative reactions. This presentation will discuss best practices in communicating user research findings to avoid these problems and to lead to better outcomes.
Not quite an Executive summary, is it?Working on this presentation and reviewing a lot of my past deliverables, I found a lot of mistakes, like this, that I made.
Any good war stories – unusual things that have happened?
This often happens when you have people observing testing. They discuss the findings as they happen, make conclusions, and decide on changes – often while you’re still testing.
[Ask the audience and raise my own hand]
[Ask the audience and raise my own hand]
[Ask the audience and raise my own hand]
[Ask the audience and raise my own hand]
[Ask the audience and raise my own hand]
Of course, if you’re writing and presenting, you need good writing skills and good presentation skills.These are very important, but we could spend hours talking about each of these.
Before we talk about creating effective research deliverables, let’s first look at what can go wrong.
Ideally the reactions are positive.This was the highest compliment I received from a client after viewing a particularly controversial presentation.
Because we deliver bad news and focus on the negative, people can get defensive.
Here are some negative reactions I’ve encounteredIt’s a good idea to anticipate and know how to prevent or deal with these reactions
Some people are impatient and just want to see the designs.Have you ever found that you’re the person in the room most interested in the research findings?
Research, analysis, and creating the deliverable takes a lot of time.Without frequent communication or involvementThey can become impatient and anxious about not seeing any designs yet. What have you been doing all this time?
Show examples of the research deliverables
They spent all this money to have you tell them what they already knew.“Well then why didn’t you do something about it?”
Why this happens:They may have known these things on some level, but didn’t have them fully formed in their minds until they saw your deliverable.It’s rare that research reveals completely new information that no one knew.Although people may have known the information, there’s great value in gathering, organizing, and formally documenting the information.People don’t understand that the purpose of research is not to educate the clients, it’s to educate the designers and project team. The clients may have already known that information, but the designers have to know it.
Clarify to clients at the beginning of the project and in deliverables:Maybe you know this information, but we need to know it firsthand.There’s value in gathering, organizing, and publishing it in official documentation that can be used for this project and future projects.Explain the value that you bring to the findings. You don’t just report directly what you hear and observe.
You say that managers can’t approve multiple items at a time.Someone pounces – You’re wrong! You can approve multiple items, you just have to…
Assume that you’re supposed to be a subject-matter expert or that you’ll become one as part of the research.What participants tell you and show you is not “correct.” So by relying on them, some of your findings may be “incorrect.”Or they pounce on you because your recommendation obviously can’t be done in the current technology.You can’t double check and verify every reported issue.
Participants’ incorrect assumptions can reveal important insights. For example, a participant may tell you that there’s no way to perform a certain function in an application. That function may exist, but they just don’t know about it, which indicates a problem in itself. Explain that research findings are iterative, you welcome feedback and clarification, and you can make revisions as neededIf something isn’t technically feasible, let’s work together to find a solution that does work
“We can do that ourselves. We’re not potted plants!” was the reaction I received from an executive.
We make it look easy – anyone can observe people and ask questionsSure, they could take time to learn about it and eventually do it themselvesMost end up doing the typical BA requirements gathering – asking people what they want
Ideally, the quality and depth of your findings will speak for themselvesYou bring your skills - you don’t just observe, ask questions, and report back
“What about the other 10,000 users,” some clients ask. I’d like to say, “Okay, do you have hundreds of millions of dollars and a few years to spend doing research with all those people?” What do they propose as an alternative then? Do research with all 10,000 or do nothing?
Clients who are used to surveys and statistical significance. Don’t understand qualitative researchUnder the misconception that research involves asking people what they want.
And why qualitative research necessitates small numbers of participantsIt’s about seeing patternsGive some good analogies and examples“Tripping over a rug”
When a participant doesn’t fit in people’s preconceived notions or brings up something they don’t want to believe. The easiest thing is to dismiss them as not a typical user.
If you don’t like the bad news, it’s easy to dismiss it by blaming the participants that were recruited.
They can’t as easily argue with the participant selection if they were part of the recruiting.Video clips reinforce that a reaction wasn’t isolated to one odd participant
For example, I didn’t mention the annoying horizontal scrolling in a reporting application in my contextual inquiry findings. It wasn’t necessary to get into obvious, usability details for field studies.We were focusing on higher level issues.
For example, usability testing will provide detailed, interface-specific findings, but field studies will result in higher-level findings such as user characteristics, user needs, tasks, and the problems they face.
But complex problems often require additional research or design exploration to solve. There isn’t always an easy solution that you can describe in text in a report or presentation.
Even lazier is the reaction, “It’s a communication issue.” People don’t need to be trained, they just need to be told how to do something.
It’s best to first make the interface as intuitive and as easy to learn as possible to minimize the need for training.Usability problems only need to be fixed once, while training is a continuous process that can become expensive.
Keep your recommendations flexible. Instead of just dismissing the recommendation completely. Try to find a solution that works around the issue.
Misunderstandings about the purpose of user research, what it involves, and the results it provides. Clearly explain the user research at the start of the project.Provide informal updates throughout the project. Don’t wait until the deliverable
There are many different types of deliverables. Which to choose depends on several factors.
Interface-related issues (usability testing, heuristic evaluation, expert review) are best explained in a visual format, like a presentation or annotated designs that can show screens and directly point to problems
Field StudiesMore about user behavior, characteristics, goals, needs, etc.Not specific about an interface. Less visual.May need to provide more detailReport, personas, diagrams, etc.
You often have to serve a variety of audiences. High-level executives want an overview.Production-level people (designers and developers) need to know the details to design and implement the solutions.You may need to provide more than one deliverable or allow for progressive disclosure.
Determines the formalityClients need a more formal deliverable, whereas you can be more informal with an internal team.
Less time – more high-level and more informal deliverable. Suitable for quick rounds of iterative design and research.
More time – more detailed and more formal deliverable. Suitable when there’s no specific deadline – as in exploratory research, provide a lot of time.
After you deliver the research findings, will you remain on as a member of the design team? If so, you can provide the knowledge. People don’t have to rely on the deliverable.
Or will you go away, and the deliverable will remain as the holder of the research knowledge? If so, you need a more comprehensive, detailed research deliverable.
There are various types of deliverables, from no deliverable at all to a full report. You should choose the deliverable that best allows you to communicate your findings and recommendations in the time available and considers the needs of your audience.
If the right people are involved throughout the research, a deliverable may not be necessary.You can do a group analysis and discussion, without creating a deliverable.They learn about the research findings and recommendations first-hand.
No one can refer to the research for questions or future projects.What happens when people leave? If the knowledge is only in their heads.Have you ever had to take over a project from another researcher? It’s hard to catch up without documentation. Looking at a bunch of random notes in notebooks and spreadsheets and various places isn’t the same as being able to refer to a deliverable
People call this various names – top-line report, summary of findings, etc.You do a quick analysis and produce an informal document with headings and bullet points describing the overall findings and recommendationsTo produce quick findingsGo through your discussion guide and write down your overall conclusions from each task or question. You’ll be surprised how much you remember off the top of your head.
Details can get lost, since they’re not documented anywhere.You may make conclusions based on a quick, high-level analysis and miss things
It’s easier to explain and for the audience to understand problems by pointing to them on an image.
It doesn’t work for research such as field studies in which the information is more about people and their tasksIt’s more difficult to present than a PowerPoint presentation – you can’t just read through it, but you can summarize
Best format to explain complex issuesSomeone who wasn’t in the presentation can read the report and understand the research. Someone joining a project later can get up to speed.Long after much of the information is forgotten, a report lives on and can be referred back to. It can be printed and it tends to stick around, whereas digital documents tend to get forgotten.A high-level PowerPoint or a brief summary of findings usually doesn’t cut it. The hearty thwack that comes from slapping down a thick report on a conference room table is impressive, even if no one reads it. To some people, a hefty and detailed report says, “Here’s some impressive, comprehensive research. We got our money’s worth on this one.”
People think of reports as these long, text-based documents, but they can really be a container for anything.
People assume they’ll be long and boring (even if they aren’t) and don’t want to read them. A presentation, you’re given the information in a specific period of time. You have to make an extra effort to read a report. And people are busy.You can’t present a report, yet you need to present something. So you end up creating both a report and a presentation, which takes even longer.Clients often want to save money by eliminating reports
A presentation can either accompany another deliverable (such as a report), or it can be the only deliverable.
It makes sense when you present it, but it doesn’t necessarily make sense to those who weren’t at the presentation.
It doesn’t have to try to contain all the details. It can leave the details for the report.Those who want more detail can refer to the report.Those who miss the presentation can read the report.
Takes more time and is more expensivePeople often don’t read a report after attending the presentation. So they don’t learn the things that aren’t in the presentation. So be sure to include the most important information in the presentation, and indicate at the end what other information exists in the report.
Has to do a lot more explaining to make sense as the only deliverable.
More people are likely to attend a presentation than to read a report
It’s difficult for presentations to contain enough detail to be understood on its own later and also be an ideal format for presenting.The extra explanatory text that makes a presentation understandable on its own doesn’t necessarily make a presentation a compelling experience when you deliver it to an audience.
Leaving out information means things can get lost, findings and recommendations can get misinterpreted
Provide an appendix of additional material that you don’t present but leave at the end of the presentation for people to read later.
Provide an appendix of additional material that you don’t present but leave at the end of the presentation for people to read later.
Create a version that you presentAnd another that you leave behind to read later and for those who didn’t attend the presentation
Create a version that’s ideal for presenting Record the presentation with Morae or Camtasia, which records the screen, your narration, and the video/audio clips; so that others can view the presentation later and get all the details as if they were at the initial presentation.
Annotated designs allow you to provide more detail
Once your audience has the initial understanding, all the extra context and examples get in the way when project teams simply want to view and discuss your recommendationsFindings and Recommendations MatrixProvides an easy way to go through the list of recommendations in a table (Word or Excel)Can just focus on the recommendationCan see the priorityOr review a summary of the findings that led to that recommendation
Easier to write than a report
These are sometimes better ways to explain and explore the findings besides simply using text or verbal description.
They usually don’t stand alone. So a report or presentation is usually also necessary.
If necessary – ask show of hands for reports, presentations, both
Regardless of the type of deliverable you provide, there are things you can do create effective deliverables
Without a template, it’s easy to waste a lot of time futzing around with formatting. You end up re-creating the wheel every time.It’s also good to maintain consistency.
It could be a great deliverable, but if it looks boring and cumbersome to read, fewer people will read it.Looking good isn’t enough, but it’s a start to get people to begin reading it or begin paying attention.
It could be a great deliverable, but if it looks boring and cumbersome to read, fewer people will read it.Looking good isn’t enough, but it’s a start to get people to begin reading it or begin paying attention.
It’s easy to overlook and think that everyone understands the methods.Even if you’ve already explained it:People forgetSome never understood it in the first placeSome weren’t involved earlier in the project. Some will read it later and need to understand
Our goal is to improve the UX, we’re often negative – finding problemsWe take the positive aspects for grantedCan be depressing and defeating for the audienceBalance the negative with some positiveExplain why you focus more on the negative
Use visuals to help the audience understand the findings – screenshots, diagrams, illustrations, charts, etc.They also break up the text, making the deliverable appear more interesting and less intimidating to read
Use visuals to help the audience understand the findings – screenshots, diagrams, illustrations, charts, etc.They also break up the text, making the deliverable appear more interesting and less intimidating to read
Use visuals to help the audience understand the findings – screenshots, diagrams, illustrations, charts, etc.They also break up the text, making the deliverable appear more interesting and less intimidating to read
Use visuals to help the audience understand the findings – screenshots, diagrams, illustrations, charts, etc.They also break up the text, making the deliverable appear more interesting and less intimidating to read
The environment, documents, tools, signage, and other physical aspects you encountered
Participants say things that we could never say.And they often have far more impact than we would have saying something general like, “The participants don’t like using the application.”
It’s easy to get carried away when you have many really great quotes.If you have too many quotes, it’s overwhelming. People won’t read them and they lose their impact.
You have more room in reports to include multiple, longer quotes.
People stop and read the quotes.Do you read the quotes to them, or do you pause and let them read them? Either way is awkward.If you have many quotes, create two versions of the presentation. One to present with only a few quotes, and one with more quotes as a read-only version.
If you have many quotes, create two versions of the presentation. One to present with only a few quotes, and one with more quotes as a read-only version.
When you can, it’s usually much more effective to show participants saying and doing things themselves.Especially for controversial findings. Difficult to argue with.
Difficult to describe complex design changes with text alone. Leaves room for misinterpretation.Can do this by making simple changes to a screenshot
Show examples from other interfaces that do what you’re recommending.
Show examples from other interfaces that do what you’re recommending.
Prioritize the recommendations – as shown in a findings and recommendations matrixOtherwise, people can be overwhelmed with the number of changes to makeWhen I was new to the field, I thought that clients should fix everything – why wouldn’t they fix it?Clients have a limited amount of time, money, and resources. Point out where they should focus first?
And give them a plan to implement the recommendationsWithout a plan, it’s easy for items to get shelved and eventually forgotten
Guide them on any additional research or design steps they need to take next.
It’s always good to get a second opinion.Get feedback from someone on your project team to ensure that your deliverable is understandable.Get a content review from a writer or editor. Not just proofreading, but ensuring that you have a well-balanced, persuasive piece of work.
Get feedback from someone on your project team to ensure that your deliverable is understandable.Get a content review from a writer or editor. Not just proofreading, but ensuring that you have a well-balanced, persuasive piece of work.
Decision makers – those with the power to make sure things get doneThe doers – those who will actually implement the recommendationsThose who don’t attend, probably won’t read the deliverable later.
This presentation is about effectively communicating user research findings. So we assume that means deliverables, but there are other ways to communicate user research findings.
“Deliverable” implies that the researcher:Conducts the research alone, while the client/project team waitsDelivers the findings and recommendationsGoes away, leaving the client/project team to implement the recommendations.
Few people will reread them after you’re gone.It’s easy to forget about or misinterpret recommendations.
Involving the project team and clients in the research gives them:More empathyA deeper understandingBetterappreciation of the findingsA stronger sense of responsibility to solve the problems
PlanningParticipants to recruitQuestions to ask, things to observe, ask the team what they want to knowHave them review and provide feedback on the discussion guide
PlanningParticipants to recruitQuestions to ask, things to observe, ask the team what they want to knowHave them review and provide feedback on the discussion guide
PlanningParticipants to recruitQuestions to ask, things to observe, ask the team what they want to knowHave them review and provide feedback on the discussion guide