SearchLove London 2018 - Els Aerts - User Research Done Right
•Download as PPTX, PDF•
1 like•2,707 views
User research, understanding your users and finding out what makes them tick, is crucial to driving growth. In this talk, Els will share some of her favourite tools and showcase where user research made all the difference.
Recommended
Recommended
More Related Content
What's hot
What's hot (19)
Similar to SearchLove London 2018 - Els Aerts - User Research Done Right
Similar to SearchLove London 2018 - Els Aerts - User Research Done Right (20)
More from Distilled
More from Distilled (20)
Recently uploaded
Recently uploaded (15)
SearchLove London 2018 - Els Aerts - User Research Done Right
- 1. Without Research There is Nothing Els Aerts #SearchLove @els_aerts
- 2. User Research Since 2001 Usability & Information Architecture Conversion optimization
- 3. Why do user research?
- 5. It depends.
- 6. You have to find out what works best for your website @els_aerts
- 7. You have to find out what works best for your audience @els_aerts
- 8. You have to find out what works best for your product or service @els_aerts
- 9. Optimizing without doing user research = Gambling @els_aerts
- 10. How much research should you do?
- 11. Just in time. Just enough.
- 13. Research is a business tool to help you make better business decisions.
- 14. Lots of methods & tools Google Analytics Scroll heatmaps User session recordings Click heatmaps Form analysis Targeted surveys 5 second tests Interviews Moderated user testing First click tests Tree structure tests Online card sorting Unmoderated user testing
- 15. Situation Quantitative research Qualitative research • Research question • What? • How many? • Why? • Type of research • Interviews • Moderated user testing • Surveys with open questions • … • Logfile analysis • Heatmaps • Form analysis • Surveys with closed questions • … • Amount of data • Small • Large • Kind of data • Unstructured data • Structured data
- 16. Situation Quantitative research Qualitative research • Research question • What? • How many? • Why? • Type of research • Logfile analysis • Heatmaps • Form analysis • … • Surveys • Interviews • User testing • … • Amount of data • Small • Large • Kind of data • Unstructured data • Structured data
- 17. Situation Quantitative research Qualitative research • Research question • What? • How many? • Why? • Type of research • Logfile analysis • Heatmaps • Form analysis • … • Surveys • Interviews • User testing • … • Amount of data • Large • Small • Kind of data • Unstructured data • Structured data
- 18. Situation Quantitative research Qualitative research • Research question • What? • How many? • Why? • Type of research • Logfile analysis • Heatmaps • Form analysis • … • Surveys • Interviews • User testing • … • Amount of data • Large • Small • Kind of data • Structured data • Unstructured data
- 19. “Empathy is seeing with the eyes of another, listening with the ears of another, and feeling with the heart of another.” - Alfred Adler
- 21. 80% of companies say they're customer-centric Only 8% of their customers agree Research by Bain & Company, 2005
- 22. User research helps you close this gap
- 23. In-depth knowledge of your customer
- 24. I’m not talking about sociodemographic data
- 25. Their motivation
- 26. Their emotions
- 27. Jeff Bezos, CEO Amazon Obligatory Amazon slide
- 30. More than 2 out of 3 visitors already use the free SEO plug-in.
- 32. Product page for newbies • Via website • Via Google Reasons to upgrade page for users of the free version • Via plugin • Via campaigns
- 33. Exit survey on product pages What’s keeping you from buying the Premium seo plug-in right now?
- 34. What’s keeping you from buying the Premium seo plug-in right now?
- 35. Exit survey on thank you page
- 36. @agconsult For newbies For people who already have the free version
- 41. 3. In-person moderated user testing @els_aerts
- 42. One on one
- 45. A focus group is not a user test.
- 46. In person moderated user testing Observation room Your team Recording the test Sharing the screen Testing room Moderator + Test participant
- 48. 1. Recruit the right users
- 49. 2. Write a good scenario
- 50. “First rule of usability? Don’t listen to users.” Jakob Nielsen
- 51. Say what now?
- 52. 3. Be a kick-ass moderator * * This is MUCH harder than it sounds.
- 53. The moderator is the biggest strength and biggest weakness in moderated user testing. @els_aerts
- 54. “A good moderator adopts 3 personalities simultaneously.” Jared Spool
- 56. 2. Sportscaster
- 58. 4. Switzerland
- 59. 99% of the talking during a user test should be done by the test user. @els_aerts
- 60. SHUT THE F*CK UP!
- 61. Thank you very much! @els_aerts
Editor's Notes
- Hi, I’m very happy to be here today to talk about something I’m absolutely passionate about. And that is user research.
- User research has been at the heart of all information architecture and all optimization projects I have done since 2001.
- Now some of you are probably thinking: “My God, woman, if you’ve been in this business since 2001, why do you still need to do research? Surely there are rules about usability and conversion rate optimization? Don’t you know them by now?
- Yeah, well… I do know these rules and guidelines. I’m sorry about the slide by the way. I tried to find a Mrs. Know it all but apparently it just doens’t exist. Anyway, sure there are there are guidelines and best practices. And a lot of those hold true for a lot of websites. But not all of them and not for all websites and all audiences all of the time. Very often still, I have to answer questions with that dreaded response…
- Every website is different. It depends on what you sell, who you are as a brand and who your audience is. Doing research, getting to know your users, finding out what makes them tick, is a vital part of any project.
- If you don’t do your research, you’re basically just gambling. You might get lucky once or twice but that won’ last.
- So how much research should you do? Should you always start off with a really long research phase before you do anything else?
- You should do just enough research for that particular moment in your project pipeline. There is always a trade off between learning and earning. So make sure you strike a balance between those two. Try and find some low-hanging fruit you can capitalize on quickly. That can buy you do the time to do the research you need for more in-depth improvements.
- A lot of people see user research as a necessary expense. That ‘s a very negative way of looking at it. You do it because it helps you make better business decisions. It helps you mitigate risk. It helps you prioritize your optimization efforts.
- There are loads of methods tools out there today. And that’s great.
- There are of course quantitative and qualitative research methods. To be clear: you need both. Quant: what and how many. Qual: why and how But there’s something else only qualitative research can teach you. (Try showing one thing at a time)
- There are of course quantitative and qualitative research methods. To be clear: you need both. Quant: what and how many. Qual: why and how But there’s something else only qualitative research can teach you. (Try showing one thing at a time)
- There are of course quantitative and qualitative research methods. To be clear: you need both. Quant: what and how many. Qual: why and how But there’s something else only qualitative research can teach you. (Try showing one thing at a time)
- There are of course quantitative and qualitative research methods. To be clear: you need both. Quant: what and how many. Qual: why and how
- And that’s empathy. All this talk of numbers and users can sometimes make you forget we’re dealing with people. Qualitative user research helps to remind you of that. It helps you to put yourself in your user’s shoes. I’ll show you some examples of how we’ve used surveys, interviews and modertaed user testing in our optimization process.
- This is a picture of the observation room during a user test we did a while ago for a Dutch department store. Look at those faces. At this point in the test, the user is struggling. And you can see it in the faces of our client. They’re suffering too! They actually said things like ‘This is really painful.’ ‘This is just horror.’ They’ll take that home with them, the awful experience that user had on their site, that empathy with the user. Link with customer centricity.
- Wie heeft dat gezegd? Attribute quotes.
- Wie heeft dat gezegd? Attribute quotes.
- Wie heeft dat gezegd? Attribute quotes.
- Wie heeft dat gezegd? Attribute quotes.
- Wie heeft dat gezegd? Attribute quotes.
- Always start with the customer. Like Jeff Bezos says: Because ultimately, they are the ones driving your growth. I’ll talk about 3 methods we use to get to know our clients’ users better and how those insights helped increase conversions. I’ll talk about 3 methods we use to get to know our clients’ users better and how those insights helped increase conversions.
- We kick off every information architecture project and almost every conversion optimization project with our top task survey. A really simple survey that appears as soon as people land on the site. usually with just 1 or 2 questions: Who are you? And What is the purpose of your visit to this website today? This gives us an insight into who visits the website and what their top tasks are on any given entry page on the site.
- We did this survey as well for our client Yoast. Where we changed the first question slightly from just ‘Who are you?’ to ‘Do you have the Yoast SEO plugin’. For a plug-in that’s installed on over 8 million sites world-wide, that’s a fair question to ask. Survey ran for 1 week A little over 7000 answers (10% response rate)
- More than 2 out of 3 visitors already use the free plug-in.
- Knowing that, it’s a bit weird to send these users of the free plug-in to this page. Bacuse this page essentially explains what the plug in is and it even has a download link for the free version. So we made some changes.
- We kept the existing product page on the Yoast website and created a separate landing page where we directed all the traffic that came from the plug-in. On this second page, we didn’t explain what th eplugin was any moe but focused on the reasons to upgrade. This change alone create an uplift of around 15% more premium sales, without hurting the number of free downloads. But we didn’t stop there. Because this second page we made based on input from the Yoast team and our own comparisons of the free and Premium version. Now we had this new landing page, we did more surveys.
- People who left the page without ordering we asked these questions.
- These questions help us get an insight into what’s holding people back. What have we maybe not explained well enough?
- We also asked people who did buy some questions. Because these people can tell us what actually are the main resaons to go Premium. We ran this for 2 weeks and got over 2000 answers.
- Various iterations of AB-testing on both pages over the past year mean that we are at a current uplift in sales for the premium plug-in of over 75%.
- A second qulitative user research method we use is interviews. Now, interviews are most valuable when we’re talking about a more complex product or service.
- We did phone interviews for Zembro, a personal alarm for seniors. We called not just seniors but also their grown children, who are usually the ones buying the alarm for the parents. It’s ajoint decision between these two parties. Again, this is something that quant could never get. (Mention empathy again.)
- 2x as many visitors scrolled down to the brochure part at the bottom of the page. +82% more brochure downlodas Paying attention to people’s concerns works.
- On to one of my favourite methods of qualitative research: in-person moderated usertesting.
- In moderated user testing empathy is crucial. The set up in this type of testing is one-on-one: there’s one moderator interacting with one test participant at a time. The individual set-up is very important. Which is why I get … slightly annoyed when people talk about focus groups as user testing. NO. That is not user testing.
- NO.
- That is fucking amateur hour.
- A focus group is not a user test. Focus groups are great as a source of idea generation. No more, no less. When it comes to usability or user testing, they are crap. User testing is one on one.
- The typical set-up for moderated user testing is the 2 room set-up. There’s the participant and moderator in the testing room, where everything is recorded on the test laptop. And there’s the observation room, where the test participants screen and audio is shared. A benefit of this set-up is you can do eye-tracking.
- We use it whenever we do in-person user testing. Here you can see the red lines on the screen where the user is looking. The red dot is where the gaze is held steady. The bigger the dot, the more time is spent looking at that area. Now, eyetracking is especially interesting for the people in the observation room. It makes the test a lot easier to follow. But it’s not a must-have. It’s the cherry on the cake. The real inisghts don’t come from the eye tracking lines or heatmaps. They come from observation.
- In moderated user testing there are 3 things that are crucial for getting it right: Recruit the right users Write a good scenario Be a kick-ass moderator *
- Let’s start with recruitment. You need to test with the right users. Age, income bracket and gender are all less important than affinity with your product or service. Make sure to get people in who are actual or potential customers. People who are interested in the stuff you’re selling. If you don’t test with the right users you ca pollute your data just like with qant.
- So you’ve got the right users. Great. Time to write your scenario. Always ask questions that lead to facts, not opinions. Even better: don’t ask questions, set tasks. You want your participant mainly to do something, not give their opinion.
- Usability guru Jakob Nielsen got it right when he said this back in 2001 “First rule of usability. Don’t listen to users”.
- Now, I can usually see some frowns at this point. Hold up: you’re talking about how we need to empathize with our users and now you’re telling us we shouldn’t listen to what they say? Yes. And no. When users start giving out ‘usability advice’ like (“I think this page would look much better with a video background.”) or when they talk about future behaviour (“I think I’d use this site a lot more if it weren’t pink.”) that’s when you need to tune out. If you want to combine the user test with an interview, that’s fine. Just know when you want the user’s opinion (on branding, on product range, on decision making criteria) and hen you want facts (on how they use your website). All of this is also why you need to be…
- …a kick-ass moderator. Fair warning: this is much harder than it sounds.
- The moderator is both the biggest strength and weakness in moderated user testing. If you’ve got a great moderator, you’ll get great inisghts. If your moderator sucks, you’re doomed.
- Usability legend Jared Spool says a good moderator combines 3 personalities in one.
- You have to establish a rapport with the test user. Make sure they’re comfortable, at ease. Make small talk when they enter the room. Offer them a drink. Check whether they’re left or right-handed. Make sure the test participant feels relaxed in your presence.
- You also have to be a sportscaster, a reporter. You’re there in the room with the test participant so you have a front row seat. Which means you sometimes see things people in the observation room can’t. For example: Sometimes a test user points at something on the screen for example without using his mouse. ‘I don’t know what that blue thing does?’ Then it’s your job to say something like: ‘Do you mean that blue image in the right hand bar? Could you show me with your mouse?’ Phrase it as a question. That way, you’re checking whether you got it right and you have brought the observation room up to speed.
- You’re not doing this just for fun. You’re here to gather data. Taking notes, writing down observations is very hard for beginning moderators. If you’re starting out, make sure you always have someone as an observer who has your back. Now, I know Jared Spool talks about 3 personality traits but I’d like to add a fourth.
- You also have to be as neutral as possible. Be mindful of your facial expressions, the way you sit, the way you breathe, the noises you make. This is probably the hardest bit. Even experienced moderators trip up sometimes..
- A good moderator lets the user take control, takes a backseat and only intervenes with extra questions when it is necessary. 99% of the talking during a user test should be done by the test user. What you as a moderator mainly need to do, is …
- … observe. And on that note that’s what I’ll do.