How to avoid writing leading survey questions
•Télécharger en tant que PPT, PDF•
6 j'aime•892 vues
https://lovestats.wordpress.com/services/ Learn the tips and tricks to writing questions that generate valid answers, as opposed to answers that seem 'correct' to responders
Recommandé
Recommandé
Contenu connexe
Plus de Annie Pettit, Research Methodologist
Plus de Annie Pettit, Research Methodologist (20)
Dernier
Dernier (20)
How to avoid writing leading survey questions
- 1. @PeanutLabsMedia How to avoid writing leading questions bids@peanutlabs.com @PeanutLabsMedia
- 2. @PeanutLabsMedia Leading Questions: Are you in market research? What industry do you work in? oAdvertising oEconomics oMarketing oJournalism Software programmer in a global company oOther What industry do you work in? Advertising Economics Journalism oMarket research Medicine Other Do you work in market research? oYes No
- 3. @PeanutLabsMedia Fixing a simple leading question • No single option stands out • Let’s not fool ourselves, market researchers will choose how to answer the market research question What is your career? oCustomer service oHealth economist oJournalism and communication oMarketing analyst oMarket research Software programmer oOther
- 4. @PeanutLabsMedia Leading Questions: Do you qualify? Did you go to the movies in the last month? Yes oNo Have you donated to the Red Cross in the last year? Yes oNo Is your income over $50 000 per year? Yes oNo
- 5. @PeanutLabsMedia Fixing a simple leading question • No single option stands out • Everyone can choose an option that makes them feel good. Which of these did you do in the last year? Bought Starbucks coffee for friends oDonated to the Red Cross oVisited a park Went to the library oNone
- 6. @PeanutLabsMedia Loaded questions: Do you like killing people? In your opinion, is the death penalty (execution) appropriate for people who have been convicted of murder? oNever oSometimes oAlways Have you struggled to sleep at night with the idea that one of those executed people might have been innocent? Yes oNo • You can’t swap sides to fix the question – Are you in favour of executing serial murderers, cop killers, and baby killers? • Yes • No
- 7. @PeanutLabsMedia Loaded questions: Do you like taxes? Which one of these would you do to improve fiscal management? oDecrease public transit oIncrease personal taxes oProvide fewer health care services oProvide fewer recreational activities Should increased taxes as part of good fiscal management be approved? Yes oNo • Remove the emotion • Provide multiple issues together • If you must address one specific issue, save it for after the unaided section
- 8. @PeanutLabsMedia Loaded questions: Do you like horrible politicians? Assuming these candidates were running for office today, which one would you vote for? oAmanda Miller oChristopher Williams oDavid Garcia oJason Jones oJennifer Johnson oJessica Rodriguez oMelissa Brown oMichael Smith Would you vote for a political candidate who has a criminal record for the sake of removing an incompetent incumbent from office? Yes oNo • Don’t scoot around the real question • If you want to know voting intentions, ask voting intentions
- 9. @PeanutLabsMedia Tips For Success Use multiple choice Provide alternative acceptable answers Make sure a desired answer doesn’t stand out 1 Remove the emotion Get close to the behavior Do a self-check 2 3 4 5 6
- 10. @PeanutLabsMedia Thank you! Annie Pettit Chief Research Officer Peanut Labs annie@peanutlabs.com twitter.com/LoveStats ca.linkedin.com/in/anniepettit/ Questions about our Sample Services? Jonathan Cheriff Director of Sales & Marketing jonathan.cheriff@peanutlabs.com twitter.com/paperbackdad www.linkedin.com/in/jonathancheriff
Notes de l'éditeur
- Welcome to the PeanutLabs quick tips for survey design webinar. There’s an old saying that you can lead a horse to water but you can’t make it drink. Sadly, you can write a leading question and make people answer it the way you want. Today, I’m going to show you a few ways to avoid writing leading questions.
- Here are some examples of the most common leading question on market research surveys. In the first two cases, everyone in market research knows what’s going on. The study manager wants to exclude people who work in market research. It’s easy enough to say, I don’t work in market research. It’s obvious which response matters. Now look at the third question. In this case, it’s completely obvious that the project manager is looking for people who are software programmers in a global company. If I want to get the incentive for this survey, I’m just going to pick the answer that sticks out the most. All of these questions are leading horses to water and probably getting them to drink.
- Fortunately, this career and industry question is easy to fix. First, there’s no point trying to disguise the market research component. We all get it and we’ll answer that part however we choose. However, the rest of the question matters a lot more. None of these answer options stand out. None of them are extra short or extra long. None of them are extra specific or extra vague. By mixing our desired answer option in with several other similar answer options that are just as vague or just as long or just as unusual, the desired answer doesn’t stick out like a sore thumb.
- Here are some more common leading questions that you might see on any market research questionnaire. Given that many people taking surveys are members of research panels, they know how these kinds of questions work. They know that if they yes, and they will probably qualify to answer the rest of questionnaire. Say no, and you might be screened out and you won’t get the incentive. In other cases, like talking about charitable behaviors, giving a yes answer makes you feel better about yourself and is less embarrassing. And of course, in some cases, it’s just easier to say yes to something you do all the time even if you didn’t do it during that time frame. All of these yes no questions make people feel like they should just cooperate by saying yes.
- But we can easily fix all of those problems. First, instead of asking “Did you donate to the Red Cross last year,” let’s include a few other options as well so that nothing in particular stands out. Now, everyone can give a yes answer and feel like they’ve contributed. And, everyone can give an answer that makes them feel better about themselves. Buying a coffee for friends feels just as nice as donating to charity. This is a great fix.
- This loaded question is really tough. We were almost forced into saying we have trouble sleeping at night because an innocent person might be executed. Now, you could ask that question from the other side and remind people that murderers of babies and cops are the focus. But that’s also a leading question. Chances are what you REALLY want to know is the percentage of people who are completely against the death penalty and won’t change their mind no matter what qualifications you make. This will determine what your next steps will be, how to handle a very sensitive topic.
- In this example, we’ve made it really difficult to say that increased taxes are a bad thing. In fact, we’ve just said it’s good thing. People will want to yes to show that they are in favour of good fiscal management. However, if we changed the question to a multi-select, then we can see how many people would choose increased taxes as their preferred method of improved fiscal management.
- Here is the last example we’ll look at. In this case, we’re being asked whether we’d like to keep an incompetent candidate in office. Well, of course we don’t. We’re being led to the yes answer. But this is a great example where we don’t need to ask about theory. We can ask exactly what we want to know. Who will you vote for. All of the emotion is gone in favour of behaviors.
- So here are a few tips to remember. Multiple choice is often your friend. Instead of yes no questions, see if it makes sense to ask it as a multi-select. And, when you do change it to a multi-select, make sure the alternative answers are similarly good or similarly bad. And, make sure that your answer doesn’t stick out, whether it’s shorter, longer, detailed, or generic. Forth, remove the emotion. You might have strong feels about a topic but make sure the nouns verbs and adjectives you use don’t. The best way to do this is by focusing on behaviors. Will you buy this, will you vote for her. And lastly, answer the question in every possible way. Figure out if one of your answers makes you feel bad or uncomfortable. Then fix it so you don’t feel bad either way.
- Thanks for listening. If you have any questions, feel free to get in touch with us. We’d love to hear from you.