SlideShare une entreprise Scribd logo

Pete cape april lecture series - 2014

•
•
Ray Poynter
Ray Poynter

This document summarizes Pete Cape's lecture on questionnaire design. It discusses various types of questions that can be included in a questionnaire, such as open-ended, yes/no, single coded, and ranked questions. It also covers important principles for questionnaire design, such as remembering that respondents want to please the researcher and may be subject to biases like acquiescence bias. Finally, it addresses best practices for measuring concepts like frequencies and attitudes to obtain accurate responses.

Marketing
1  sur  185
Télécharger pour lire hors ligne
Pete Cape, SSI, UK April Lecture Series 2014 Questionnaire Design – A Masterclass Pete Cape Global Knowledge Director, SSI 2014 Platinum Sponsor April Series Sponsor Lecture Sponsor
Pete Cape, SSI, UK April Lecture Series 2014 | 2 | A questionnaire might have all of these…. • Open questions • Yes/No questions • Single coded questions with multiple answers • Multi-coded questions with multiple answers • Items to be ranked • Items to be rated • Items to be chosen between
Pete Cape, SSI, UK April Lecture Series 2014 | 3 | A questionnaire might have all of these…. • Open questions • Yes/No questions • Single coded questions with multiple answers • Multi-coded questions with multiple answers • Items to be ranked • Items to be rated • Items to be chosen between • But this is such a boring way of looking at it….
Pete Cape, SSI, UK April Lecture Series 2014 | 4 | A questionnaire might have all of these…. • Open questions • Yes/No questions • Single coded questions with multiple answers • Multi-coded questions with multiple answers • Items to be ranked • Items to be rated • Items to be chosen between • But this is such a boring way of looking at it…. • Let’s start with some principles….
Pete Cape, SSI, UK April Lecture Series 2014 | 5 | Things to remember about respondents….
Pete Cape, SSI, UK April Lecture Series 2014 | 6 | Things to remember about respondents…. • They know they are in an experiment
Pete Cape, SSI, UK April Lecture Series 2014 | 7 | Things to remember about respondents…. • They know they are in an experiment • Everything they see, hear or read is meant deliberately, and has meaning
Pete Cape, SSI, UK April Lecture Series 2014 | 8 | Things to remember about respondents…. • They know they are in an experiment • Everything they see, hear or read is meant deliberately, and has meaning • They want to please
Pete Cape, SSI, UK April Lecture Series 2014 | 9 | Things to remember about respondents…. • They know they are in an experiment • Everything they see, hear or read is meant deliberately, and has meaning • They want to please • They think you know what you are doing…
Pete Cape, SSI, UK April Lecture Series 2014
Pete Cape, SSI, UK April Lecture Series 2014
Pete Cape, SSI, UK April Lecture Series 2014 | 12 | Things to remember about questionnaires…. • Everything has meaning
Pete Cape, SSI, UK April Lecture Series 2014 | 13 | Things to remember about questionnaires…. • Everything has meaning • You are supposed to know what that meaning is
Pete Cape, SSI, UK April Lecture Series 2014 | 14 | Questionnaires do a lot of work • Motivate the respondent
Pete Cape, SSI, UK April Lecture Series 2014 | 15 | Questionnaires do a lot of work • Motivate the respondent to provide — complete answers — accurate answers
Pete Cape, SSI, UK April Lecture Series 2014 | 16 | Questionnaires do a lot of work • Motivate the respondent to provide — complete answers — accurate answers • Communicate to the respondent
Pete Cape, SSI, UK April Lecture Series 2014 | 17 | Questionnaires do a lot of work • Motivate the respondent to provide — complete answers — accurate answers • Communicate to the respondent • Help the respondent work out their answers
Pete Cape, SSI, UK April Lecture Series 2014 | 18 | Questionnaires do a lot of work • Motivate the respondent to provide — complete answers — accurate answers • Communicate to the respondent • Help the respondent work out their answers • Make the respondents task easy
Pete Cape, SSI, UK April Lecture Series 2014 | 19 | Is this the simplest question of all? • Yes • No
Pete Cape, SSI, UK April Lecture Series 2014 | 20 | Respondents want to please Do you care to disagree with me….? I am a scientist….. • Acquiescence bias
Pete Cape, SSI, UK April Lecture Series 2014 Acquiescence bias • Some respondents are simply agreeable, and indicate agreement out of politeness. Saris, Krosnick and Shaeffe
Pete Cape, SSI, UK April Lecture Series 2014 Acquiescence bias • Some respondents are simply agreeable, and indicate agreement out of politeness. • Other respondents expect that the researchers agree with the listed items and defer to their judgment. Saris, Krosnick and Shaeffe
Pete Cape, SSI, UK April Lecture Series 2014 Acquiescence bias • Some respondents are simply agreeable, and indicate agreement out of politeness. • Other respondents expect that the researchers agree with the listed items and defer to their judgment. • Most respondents engage in survey satisficing and find that agreeing takes less effort than carefully weighing each optional level of disagreement and agreement. Saris, Krosnick and Shaeffe
Pete Cape, SSI, UK April Lecture Series 2014 Affirmation vs Multicode
Pete Cape, SSI, UK April Lecture Series 2014 Affirmation vs Multicode
Pete Cape, SSI, UK April Lecture Series 2014 Overcoming acquiescence bias • Find them through traps and exclude
Pete Cape, SSI, UK April Lecture Series 2014 Overcoming acquiescence bias • Find them through traps and exclude • Follow up question “how sure?” • If not 100% sure then recode as “no”
Pete Cape, SSI, UK April Lecture Series 2014 Overcoming acquiescence bias • Find them through traps and exclude • Follow up question “how sure?” • If not 100% sure then recode as “no” • Avoid the question type and give alternatives — “chose the one of these that best describes you” — “to what extent do you…”
Pete Cape, SSI, UK April Lecture Series 2014 Affirmation vs Multicode
Pete Cape, SSI, UK April Lecture Series 2014 Affirmation vs Multicode why so big…?
Pete Cape, SSI, UK April Lecture Series 2014 Social Desirability bias • Wanting to look good
Pete Cape, SSI, UK April Lecture Series 2014 Social Desirability bias • Wanting to look good — To society — To other individuals — To myself
Pete Cape, SSI, UK April Lecture Series 2014 Some Normal Everyday Things…that I keep to myself.. Chart Title
Pete Cape, SSI, UK April Lecture Series 2014 Some Normal Everyday Things…I want you to think I do.. Chart Title
Pete Cape, SSI, UK April Lecture Series 2014 20% bought a book last week? | 35 |
Pete Cape, SSI, UK April Lecture Series 2014 Avoiding Social Desirability • Is very hard • Try to make respondents feel “normal”
Pete Cape, SSI, UK April Lecture Series 2014 Avoiding Social Desirability • Is very hard • Try to make respondents feel “normal” This third section is about social attitudes and values. There is a lot of talk about discrimination and racial prejudice, and different people have different views on the size of such problems, their causes and what should be done about them. There are no right or wrong answers, it is your honest opinions we are looking for. If you do not wish to answer any of these questions please use the option “prefer not to say”.
Pete Cape, SSI, UK April Lecture Series 2014 Questionnaire Design and Bias • Questionnaire design has to overcome respondents’ psychological biases
Pete Cape, SSI, UK April Lecture Series 2014 Questionnaire Design and Bias • Questionnaire design has to overcome respondents’ psychological biases • And also experimenter bias
Pete Cape, SSI, UK April Lecture Series 2014 Questionnaire Design and Bias • Questionnaire design has to overcome respondents’ psychological biases • And also experimenter bias • i.e. you and your client….
Pete Cape, SSI, UK April Lecture Series 2014 Experimenter biases • It is easy to suggest the answer…
Pete Cape, SSI, UK April Lecture Series 2014 Question suggests the answer Which carbonated soft drinks, like Coca-Cola or Pepsi for example, do you drink regularly?”
Pete Cape, SSI, UK April Lecture Series 2014 Question suggests the answer Which carbonated soft drinks, like Coca-Cola or Pepsi for example, do you drink regularly?” Raises answers for drinks “like” Pepsi and Coke i.e. Brown or cola drinks
Pete Cape, SSI, UK April Lecture Series 2014 Question suggests the answer Which carbonated soft drinks, like Coca-Cola or Pepsi for example, do you drink regularly?” Raises answers for drinks “like” Pepsi and Coke i.e. Brown or cola drinks Lowers answers for non- brown drinks e.g. Fanta, 7-Up
Pete Cape, SSI, UK April Lecture Series 2014 Question suggests the answer Which carbonated soft drinks, like Coca-Cola or Pepsi for example, do you drink regularly?” Raises answers for drinks “like” Pepsi and Coke i.e. Brown or cola drinks Lowers answers for non- brown drinks e.g. Fanta, 7-Up Problem is in use of jargon term: “carbonated soft drinks”
Pete Cape, SSI, UK April Lecture Series 2014 Question suggests “my” answer Testing perfumes on animals involves causing them pain. Do you agree with testing perfumes on animals?
Pete Cape, SSI, UK April Lecture Series 2014 Question suggests “my” answer Testing perfumes on animals involves causing them pain. Do you agree with testing perfumes on animals? “involves causing them pain” is an opinion
Pete Cape, SSI, UK April Lecture Series 2014 Avoiding Experimenter Bias • Is easier
Pete Cape, SSI, UK April Lecture Series 2014 Avoiding Experimenter Bias • Is easier • Think neutral
Pete Cape, SSI, UK April Lecture Series 2014 Avoiding Experimenter Bias • Is easier • Think neutral • Don’t think your view is the world view
Pete Cape, SSI, UK April Lecture Series 2014 Questionnaire as a measuring tool • We rarely measure physical things
Pete Cape, SSI, UK April Lecture Series 2014 Questionnaire as a measuring tool • We rarely measure physical things • But we do measure frequency: — How often do you x, y, z?
Pete Cape, SSI, UK April Lecture Series 2014 Questionnaire as a measuring tool • We rarely measure physical things • But we do measure frequency: — How often do you x, y, z? • And we do measure attitudes: — How much do you think x,y,z?
Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • As an relative concept in a fixed time period
Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • As a relative concept in a fixed time period • Define: — The activity • What does it mean to “go shopping”? • What does it mean to “surf the internet?” — The time period • Is it the relevant one? • What is an “average week”? — The measurement scale • Always, often, sometimes, never…..
Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • As an absolute number in a fixed time period
Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • As an absolute number in a fixed time period • Define: — The activity • What does it mean to “go shopping”? • What does it mean to “surf the internet?” — The time period • Is it the relevant one? • What is an “average week”?
Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • As an absolute number in a fixed time period How many times did you take a bath in the past year?
Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • As an absolute number in a fixed time period How many times did you take a bath in the past year? too longwhat does this mean?
Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • Precision and correct framing How many times, if at all, did you take a bath in the past week? By taking a bath we mean actually using a bath tub and filling it with water and sitting in it. Please exclude any showers you may have taken
Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • Precision and correct framing How many times, if at all, did you take a bath in the past week? By taking a bath we mean actually using a bath tub and filling it with water and sitting in it. Please exclude any showers you may have taken OMG, I only take showers, he will think I am so dirty
Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • Precision and correct framing, and overcoming social desirability bias Some people only take baths, some only take showers, some take both. This part of the survey is only concerned with bath taking. How many times, if at all, did you take a bath in the past week? By taking a bath we mean actually using a bath tub and filling it with water and sitting in it. Please exclude any showers you may have taken
Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • Precision and correct framing, and overcoming social desirability bias, and to make life easy Some people only take baths, some only take showers, some take both. This part of the survey is only concerned with bath taking. How many times, if at all, did you take a bath in the past week? By taking a bath we mean actually using a bath tub and filling it with water and sitting in it. Please exclude any showers you may have taken 0 / only took showers 1 – 2 3 – 4 5 – 6 7 or more Can’t remember
Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • Precision and correct framing, and overcoming social desirability bias, and to make life easy Some people only take baths, some only take showers, some take both. This part of the survey is only concerned with bath taking. How many times, if at all, did you take a bath in the past week? By taking a bath we mean actually using a bath tub and filling it with water and sitting in it. Please exclude any showers you may have taken 0 / only took showers 1 – 2 3 – 4 5 – 6 7 or more Can’t remember Does this have any meaning?
Pete Cape, SSI, UK April Lecture Series 2014 The meaning of ranges
Pete Cape, SSI, UK April Lecture Series 2014 The meaning of ranges • Assumption of normal distribution
Pete Cape, SSI, UK April Lecture Series 2014 The meaning of ranges • Assumption of normal distribution
Pete Cape, SSI, UK April Lecture Series 2014 Manipulating numbers Answer list A Answer list B
Pete Cape, SSI, UK April Lecture Series 2014 Manipulating numbers Answer list A Answer list B
Pete Cape, SSI, UK April Lecture Series 2014 Manipulating numbers Answer list A Answer list B
Pete Cape, SSI, UK April Lecture Series 2014 Manipulating numbers Answer list A • The central banding should contain the mean/median • But we don’t know it, it’s what we’re trying to find out! Answer list B
Pete Cape, SSI, UK April Lecture Series 2014 Collecting real numbers • Just ask for it • Respondent knows it and will give it
Pete Cape, SSI, UK April Lecture Series 2014 Measuring attitudes
Pete Cape, SSI, UK April Lecture Series 2014 Measuring attitudes Fill in the words that correspond to the scale points for “Likely” 0 = not at all likely 1 = ? 2 = ? 3 = ? 4 = ? 5 = ? 6 = ? 7 = ? 8 = ? 9 = ? 10 = extremely likely ? ? Exercise
Pete Cape, SSI, UK April Lecture Series 2014 Measuring attitudes Fill in the words that correspond to the scale points for “Likely” 0 = not at all likely 1 = ? 2 = ? 3 = ? 4 = ? 5 = ? 6 = ? 7 = ? 8 = ? 9 = ? 10 = extremely likely Exercise
Pete Cape, SSI, UK April Lecture Series 2014 Measuring attitudes Fill in the words that correspond to the scale points for “Likely” 0 = not at all likely 1 = ? 2 = ? 3 = ? 4 = ? 5 = ? 6 = ? 7 = ? 8 = ? 9 = ? 10 = extremely likely Exercise Exercise outcomes: a) No-one will be able to do it b) Some people will write: 5 = “neither likely not unlikely” 5 = “some degree of likelihood”
Pete Cape, SSI, UK April Lecture Series 2014 Measuring attitudes Fill in the words that correspond to the scale points for “Likely” 0 = not at all likely 1 = ? 2 = ? 3 = ? 4 = ? 5 = ? 6 = ? 7 = ? 8 = ? 9 = ? 10 = extremely likely Exercise Exercise outcomes: a) No-one will be able to do it b) Some people will write: 5 = “neither likely nor unlikely” 5 = “some degree of likelihood”
Pete Cape, SSI, UK April Lecture Series 2014 Two men go into a bar….
Pete Cape, SSI, UK April Lecture Series 2014 Two men go into a bar…. I say, what do you think of the new VW Golf?
Pete Cape, SSI, UK April Lecture Series 2014 Two men go into a bar…. I say, what do you think of the new VW Golf? I think it’s a 6…
Pete Cape, SSI, UK April Lecture Series 2014 Two men go into a bar…. Attitudes are expressed in words not numbers I say, what do you think of the new VW Golf? I think it’s a 6…
Pete Cape, SSI, UK April Lecture Series 2014 Numbers are universal
Pete Cape, SSI, UK April Lecture Series 2014 Numbers are universal • Numeric scales are not!
Pete Cape, SSI, UK April Lecture Series 2014 Numbers are universal • Numeric scales are not! • It all starts in school — 0 = you weren’t even there! — 1-4 = you might as well not have been there… — 5-6 = you are going to have to work way harder in the future — 7-8 = okay, could do better — 9 = good — 10 = very good
Pete Cape, SSI, UK April Lecture Series 2014 Numbers are universal • Numeric scales are not! • It all starts in school — 0 = you weren’t even there! — 1-4 = you might as well not have been there… — 5-6 = you are going to have to work way harder in the future — 7-8 = okay, could do better — 9 = good — 10 = very good • And this is for cultures that score 0 - 10, does everyone?
Pete Cape, SSI, UK April Lecture Series 2014 German School System • 1 (sehr gut, very good) is the best possible grade and is given for outstanding performance • 2 (gut, good) is the next-highest and is given for performance that meets the standard completely and is above-average • 3 (befriedigend, satisfactory) indicates "average" performance. • 4 (ausreichend, sufficient) is the lowest passing grade and is given if the standard has been met but with a number of notable errors. • 5 (mangelhaft, deficient) is the higher of two failing grades and is given if the standard has not been met but the basics have been understood. • 6 (ungenügend, insufficient) is the lowest possible grade and is given if the standard has not been met and the basics have not been understood.
Pete Cape, SSI, UK April Lecture Series 2014 Numbers have meaning 0 1 2 3 4 5 6 7 8 9 10 Schwartz, 1996
Pete Cape, SSI, UK April Lecture Series 2014 Numbers have meaning 0 1 2 3 4 5 6 7 8 9 10 34% Schwartz, 1996
Pete Cape, SSI, UK April Lecture Series 2014 Numbers have meaning 0 1 2 3 4 5 6 7 8 9 10 -5 -4 -3 -2 -1 0 1 2 3 4 5 34% Schwartz, 1996
Pete Cape, SSI, UK April Lecture Series 2014 Numbers have meaning 0 1 2 3 4 5 6 7 8 9 10 -5 -4 -3 -2 -1 0 1 2 3 4 5 34% 13%? Schwartz, 1996
Pete Cape, SSI, UK April Lecture Series 2014 Numbers have meaning 0 1 2 3 4 5 6 7 8 9 10 -5 -4 -3 -2 -1 0 1 2 3 4 5 34% 13% Schwartz, 1996
Pete Cape, SSI, UK April Lecture Series 2014 This what scales are trying to achieve
Pete Cape, SSI, UK April Lecture Series 2014 This what scales are trying to achieve
Pete Cape, SSI, UK April Lecture Series 2014 • Turn to academia for the answer How long should a scale be?
Pete Cape, SSI, UK April Lecture Series 2014 • Turn to academia for the answer • Is not always shared  • But is somewhere between 5 and 9 points How long should a scale be?
Pete Cape, SSI, UK April Lecture Series 2014 • Turn to academia for the answer • Is not always shared  • But is somewhere between 5 and 9 points • Jon Krosnick says: • Unipolar: 5 points • Bipolar: 7 points How long should a scale be?
Pete Cape, SSI, UK April Lecture Series 2014 • Turn to academia for the answer • Is not always shared  • But is somewhere between 5 and 9 points • Jon Krosnick says: • Unipolar: 5 points • Bipolar: 7 points • Why? • Maximises Reliability and Validity How long should a scale be?
Pete Cape, SSI, UK April Lecture Series 2014 What is reliability and validity? • Reliability — Measures the same thing in the same way every time
Pete Cape, SSI, UK April Lecture Series 2014 What is reliability and validity? • Reliability — Measures the same thing in the same way every time • Validity — Measures what you think it measures
Pete Cape, SSI, UK April Lecture Series 2014 What was Unipolar and Bipolar? “Unipolar” One direction from zero to lots he middle and surfeit at the ends
Pete Cape, SSI, UK April Lecture Series 2014 What was Unipolar and Bipolar? “Unipolar” One direction from zero to lots “Bipolar” Opposites with indifference in the middle he middle and surfeit at the ends
Pete Cape, SSI, UK April Lecture Series 2014 Scales Unipolar: • Extremely x • Very x • Somewhat x • Slightly x • Not at all x (an) academic conclusion
Pete Cape, SSI, UK April Lecture Series 2014 Scales Unipolar: • Extremely x • Very x • Somewhat x • Slightly x • Not at all x Bipolar: • Extremely x • Very x • Somewhat x • Neither x nor un-x • Somewhat un-x • Very un-x • Extremely un-x (an) academic conclusion
Pete Cape, SSI, UK April Lecture Series 2014 Scales Unipolar: • Extremely x • Very x • Somewhat x • Slightly x • Not at all x Bipolar: • Extremely x • Very x • Somewhat x • Neither x nor un-x • Somewhat un-x • Very un-x • Extremely un-x And this is what I recommend when asked.... btw: you can’t mix bi- and unipolar scales (an) academic conclusion
Pete Cape, SSI, UK April Lecture Series 2014 Scales • Design issues • Display issues
Pete Cape, SSI, UK April Lecture Series 2014 Where to put “don’t know” • Visual and Conceptual midpoint
Pete Cape, SSI, UK April Lecture Series 2014 Where to put “don’t know” • Visual and Conceptual midpoint Conceptual midpoint Visual midpoint
Pete Cape, SSI, UK April Lecture Series 2014 Where to put “don’t know” • Visual and Conceptual midpoint
Pete Cape, SSI, UK April Lecture Series 2014 Where to put “don’t know” • Visual and Conceptual midpoint Visual midpoint
Pete Cape, SSI, UK April Lecture Series 2014 Where to put “don’t know” • Visual and Conceptual midpoint • Skews data towards “disagree” Conceptual midpoint Visual midpoint
Pete Cape, SSI, UK April Lecture Series 2014 Where to put “don’t know” • Visual and Conceptual midpoint Conceptual midpoint Visual midpoint
Pete Cape, SSI, UK April Lecture Series 2014 What order to present? Disagree Strongly Disagree Slightly Neither Agree nor Disagree Agree Slightly Agree Strongly
Pete Cape, SSI, UK April Lecture Series 2014 What order to present? Disagree Strongly Disagree Slightly Neither Agree nor Disagree Agree Slightly Agree Strongly Order bias
Pete Cape, SSI, UK April Lecture Series 2014 What order to present? Disagree Strongly Disagree Slightly Neither Agree nor Disagree Agree Slightly Agree Strongly Order bias Acquiescence bias
Pete Cape, SSI, UK April Lecture Series 2014 What order to present? Disagree Strongly Disagree Slightly Neither Agree nor Disagree Agree Slightly Agree Strongly Order bias Acquiescence bias Central tendency
Pete Cape, SSI, UK April Lecture Series 2014 What order to present? Disagree Strongly Disagree Slightly Neither Agree nor Disagree Agree Slightly Agree Strongly Order bias Acquiescence bias Central tendency Agree Strongly Agree Slightly Neither Agree nor Disagree Disagree Slightly Disagree Strongly Central tendency Order bias Acquiescence bias
Pete Cape, SSI, UK April Lecture Series 2014 Where to put the answer box • Next to answers • 22% failure rate on matching gender to panel record!
Pete Cape, SSI, UK April Lecture Series 2014 Where to put the answer box • Next to answers • 22% failure rate on matching gender to panel record! • Left right processing • 98% of that: Males coded as Females
Pete Cape, SSI, UK April Lecture Series 2014 Scales • Maximises reliability and validity • Minimises bias • Prevents order error • Not just “the way we always do it…”
Pete Cape, SSI, UK April Lecture Series 2014 Scales • Maximises reliability and validity • Minimises bias • Prevents order error • Not just “the way we always do it…” • But isn’t it old-fashioned?
Pete Cape, SSI, UK April Lecture Series 2014 Sliders with visual elements
Pete Cape, SSI, UK April Lecture Series 2014 129125 77 25 4 0 50 100 150 200 250 Different styles of slider scale
Pete Cape, SSI, UK April Lecture Series 2014 Different styles of slider scale 114 8892 17 1 0 50 100 150 200 250
Pete Cape, SSI, UK April Lecture Series 2014 Different styles of slider scale 97 21 27 8 1 0 50 100 150 200 250
Pete Cape, SSI, UK April Lecture Series 2014 4 10 124 4751 0 50 100 150 200 250 Different styles of slider scale
Pete Cape, SSI, UK April Lecture Series 2014 Different distributions, same means? UK USA Germany China traditional - raw data 2.27 2.32 2.50 3.79 traditional - rescored 2.60 2.28 2.50 3.87 slider with all labels 2.56 2.64 2.74 3.89 slider with all tick marks 2.33 2.35 2.46 3.61 slider with no tick marks 2.32 2.39 2.29 3.62 slider with no tick marks, score showing 2.34 2.44 2.33 3.56
Pete Cape, SSI, UK April Lecture Series 2014 Grids and Matrices • Q: Do we have these in telephone research? • Q: In Face to Face?
Pete Cape, SSI, UK April Lecture Series 2014 Grids and Matrices • Q: Do we have these in telephone research? • Q: In Face to Face? • A: No • Q: Where did we have them?
Pete Cape, SSI, UK April Lecture Series 2014 Grids and Matrices • Q: Do we have these in telephone research? • Q: In Face to Face? • A: No • Q: Where did we have them? • A: postal research
Pete Cape, SSI, UK April Lecture Series 2014 Item-in-a-series
Pete Cape, SSI, UK April Lecture Series 2014 Item-in-a-series “When one has to ask a series of questions that use the same answer categories it is convenient to combine them into a [grid/matrix format]. Combining them into this ‘item-in-a- series’, with a common introduction that defines the general question and response format, eliminates considerable redundancy with regard to stating questions, it also saves considerable space”
Pete Cape, SSI, UK April Lecture Series 2014 Item-in-a-series “but do it carefully”
Pete Cape, SSI, UK April Lecture Series 2014 Item-in-a-series “but do it carefully” • Items are now in a comparative framework • Visual rendering makes them a unit “if the sponsor wants individuals to contemplate each item separately, it is advisable to present each of them as individual items”
Pete Cape, SSI, UK April Lecture Series 2014 Item-in-a-series Should these should be the same?
Pete Cape, SSI, UK April Lecture Series 2014 Item-in-a-series Shouldn’t these have been the same?
Pete Cape, SSI, UK April Lecture Series 2014 | 136 | What else do grids do? • Encourage you to go too fast
Pete Cape, SSI, UK April Lecture Series 2014 Speed of grid completion •What sort of processing is being done at this speed? | 137 |
Pete Cape, SSI, UK April Lecture Series 2014 | 138 | What else do grids do? • Encourage you to go too fast • Process the underlying latent construct, not the items — Straightlining
Pete Cape, SSI, UK April Lecture Series 2014 | 139 | What else do grids do? • Encourage you to go too fast • Process the underlying latent construct, not the items — Straightlining • Miss the subtleties — Heuristic answering
Pete Cape, SSI, UK April Lecture Series 2014 Direct instruction at item 39/40 – pass/fail | 140 |
Pete Cape, SSI, UK April Lecture Series 2014 Which heuristic was used? | 141 | answer at trap
Pete Cape, SSI, UK April Lecture Series 2014 Which heuristic was used? | 142 | answer at trap
Pete Cape, SSI, UK April Lecture Series 2014 And those left over? | 143 |
Pete Cape, SSI, UK April Lecture Series 2014 And those left over? | 144 |
Pete Cape, SSI, UK April Lecture Series 2014 | 145 | Anything else wrong with grids? • Yes, we use the wrong scale for the construct
Pete Cape, SSI, UK April Lecture Series 2014 Beware of the agree-disagree question • Make your scale answers the appropriate ones To what extent to you agree or disagree that this training course is extremely useful? Agree extremely Very much agree Somewhat agree Neither agree nor disagree Somewhat disagree Very much disagree Disagree extremely
Pete Cape, SSI, UK April Lecture Series 2014 2 stage process 1. Decide how useful you find this
Pete Cape, SSI, UK April Lecture Series 2014 2 stage process 1. Decide how useful you find this 2. Translate into agree-disagree, hopefully: Extremely useful = Agree extremely Very useful = Very much agree Somewhat useful = Somewhat agree Neither useful not disuseful = Neither agree nor disagree Somewhat disuseful = Somewhat disagree Very disuseful = Very much disagree Extremely disuseful = Disagree extremely
Pete Cape, SSI, UK April Lecture Series 2014 2 stage process 1. Decide how useful you find this 2. Translate into agree-disagree, hopefully: • Disuseful? Extremely useful = Agree extremely Very useful = Very much agree Somewhat useful = Somewhat agree Neither useful not disuseful = Neither agree nor disagree Somewhat disuseful = Somewhat disagree Very disuseful = Very much disagree Extremely disuseful = Disagree extremely
Pete Cape, SSI, UK April Lecture Series 2014 2 stage process • Reality (maybe): Extremely useful = Agree extremely Very useful = Very much agree Somewhat useful = Somewhat agree Slightly useful = Somewhat agree Not at all useful = Neither/nor, Somewhat disagree,Very much disagree,Disagree extremely
Pete Cape, SSI, UK April Lecture Series 2014 How to measure attitudes/constructs • Work out the underlying construct — i.e. what are you measuring
Pete Cape, SSI, UK April Lecture Series 2014 How to measure attitudes/constructs • Work out the underlying construct — i.e. what are you measuring • Decide how it is measured — What words — Unipolar or Bipolar
Pete Cape, SSI, UK April Lecture Series 2014 How to measure attitudes/constructs • Work out the underlying construct — i.e. what are you measuring • Decide how it is measured — What words — Unipolar or Bipolar • Check your measurement gaps are equal • Ask question..
Pete Cape, SSI, UK April Lecture Series 2014 How to measure attitudes/constructs • Work out the underlying construct — i.e. what are you measuring • Decide how it is measured — What words — Unipolar or Bipolar • Check your measurement gaps are equal • Ask question.. • More work for you, less work for respondent • And more valid and reliable
Pete Cape, SSI, UK April Lecture Series 2014 Construct Specific Scale How useful, if at all, did you find this training course? Extremely useful Very useful Somewhat useful Slightly useful Not at all useful • As you read the question your answer is forming….. ?
Pete Cape, SSI, UK April Lecture Series 2014 Construct Specific Scale How useful, if at all, did you find this training course? Extremely useful Very useful Somewhat useful Slightly useful Not at all useful • As you read the question your answer is forming….. • Order effects?
Pete Cape, SSI, UK April Lecture Series 2014 Construct Specific Scale Extremely useful Very useful Somewhat useful Slightly useful Not at all useful
Pete Cape, SSI, UK April Lecture Series 2014 Construct Specific Scale Extremely useful Very useful Somewhat useful Slightly useful Not at all useful Expectation: up = good (Tourangeau)
Pete Cape, SSI, UK April Lecture Series 2014 Construct Specific Scale Extremely useful Very useful Somewhat useful Slightly useful Not at all useful Expectation: up = good (Tourangeau) Answer speed = faster (Christian et al) No order bias found
Pete Cape, SSI, UK April Lecture Series 2014 Manipulating people by accident • Because we want to make it more fun
Pete Cape, SSI, UK April Lecture Series 2014 Please select your most preferred holiday type… Summer Beach Winter City Break Adventure
Pete Cape, SSI, UK April Lecture Series 2014 Please select your most preferred holiday type… Summer Beach Winter City Break Adventure
Pete Cape, SSI, UK April Lecture Series 2014 Please select your most preferred holiday type… Summer Beach Winter City Break Adventure
Pete Cape, SSI, UK April Lecture Series 2014 The results 58% 60% 36% 0% 20% 40% 60% 80% 100% “Summer Beach”
Pete Cape, SSI, UK April Lecture Series 2014 Motivation
Pete Cape, SSI, UK April Lecture Series 2014 Motivation Self-Determination Theory (Deci & Ryan)
Pete Cape, SSI, UK April Lecture Series 2014 Motivation • people are active ‘organisms’ • having tendencies toward psychological growth and development • who strive to master ongoing challenges • and to integrate their experiences into a coherent sense of self. • the social context can either support or thwart the natural tendencies Self-Determination Theory (Deci & Ryan)
Pete Cape, SSI, UK April Lecture Series 2014 Motivation under Self-Determination Theory A continuum that relates to the task in the social context, not a person
Pete Cape, SSI, UK April Lecture Series 2014 Motivation under Self-Determination Theory A continuum that relates to the task in the social context, not a person Regulation Introjection Identification Integration to obtain an externally imposed reward pressure to avoid guilt or to enhance self-esteem conscious valuing of activity assimilated to self task is not done for it’s own reward
Pete Cape, SSI, UK April Lecture Series 2014 Motivation under Self-Determination Theory A continuum that relates to the task in the social context, not a person Regulation Introjection Identification Integration to obtain an externally imposed reward pressure to avoid guilt or to enhance self-esteem conscious valuing of activity assimilated to self task is not done for it’s own reward Where do most market research respondents fall?
Pete Cape, SSI, UK April Lecture Series 2014 Motivation under Self-Determination Theory A continuum that relates to the task in the social context, not a person Regulation Introjection Identification Integration to obtain an externally imposed reward pressure to avoid guilt or to enhance self-esteem conscious valuing of activity assimilated to self task is not done for it’s own reward Where do most market research respondents fall?
Pete Cape, SSI, UK April Lecture Series 2014 Motivation under Self-Determination Theory Task Outcomes Regulation Introjection Identification Integration little interest, value or effort more effort, anxiety, poor coping with failure enjoyment, good coping skills greater enjoyment, competency task is not done properly interest, enjoyment highly competent Where would we like respondents to be?
Pete Cape, SSI, UK April Lecture Series 2014 Motivation under Self-Determination Theory Task Outcomes Regulation Introjection Identification Integration little interest, value or effort more effort, anxiety, poor coping with failure enjoyment, good coping skills greater enjoyment, competency task is not done properly interest, enjoyment highly competent Where would we like respondents to be?
Pete Cape, SSI, UK April Lecture Series 2014 Moving people along the motivation continuum • Foster feelings of: — Autonomy – you are free to do this or not as you choose — Competence – you are good at this — Relatedness – people like you do this — Value – what you are doing has meaning
Pete Cape, SSI, UK April Lecture Series 2014 Moving people along the motivation continuum • Foster feelings of: — Autonomy – you are free to do this or not as you choose — Competence – you are good at this — Relatedness – people like you do this — Value – what you are doing has meaning • Perhaps this was what the interviewer used to do? • This is what Gamification helps do
Pete Cape, SSI, UK April Lecture Series 2014 Motivating respondents Thank you very much for choosing to do this important survey. As one of our most expert survey takers we have selected you to help on this survey. People like you all over the country are taking part in this survey. Your answers and those of everyone else will help improve the services our client provides to customers like you. Please press the > button when you are ready to continue with your survey.
Pete Cape, SSI, UK April Lecture Series 2014 Motivating respondents Thank you very much for choosing to do this important survey. As one of our most expert survey takers we have selected you to help on this survey. People like you all over the country are taking part in this survey. Your answers and those of everyone else will help improve the services our client provides to customers like you. Please press the > button when you are ready to continue with your survey. • No mention of rewards • No threats
Pete Cape, SSI, UK April Lecture Series 2014 Summary • The questionnaire is a mixture of art and science • The art of writing well, of motivating respondents • The science of writing precisely, avoiding bias • Everything has meaning • You are supposed to know what it means • There are better ways of asking questions • There are better ways of collecting answers • You might as well do the best you can
Pete Cape, SSI, UK April Lecture Series 2014 Summary • Questionnaire writing • (and understanding why it is important) • Sets us apart from others • And has real value
Pete Cape, SSI, UK April Lecture Series 2014 Summary - Principles • Respondents know they are in an experiment • They think everything they see, hear or read is deliberate, and has meaning • They want to please • Questionnaire motivates the respondent to provide — complete answers — accurate answers • It communicates to the respondent • It helps the respondent work out their answers • It makes the respondents task easy
Pete Cape, SSI, UK April Lecture Series 2014 And guess what? • If it was easy do you think there would be textbooks on it?
Pete Cape, SSI, UK April Lecture Series 2014 Thank you! Pete Cape Global Knowledge Director, SSI 2014 Platinum Sponsor April Series Sponsor Lecture Sponsor
Pete Cape, SSI, UK April Lecture Series 2014 The Sponsors for this Event If you are interested in sponsoring a future NewMR event Email Michele.Poynter@TheFuturePlace.com 2014 Platinum Sponsor April Series Sponsor Lecture Sponsor
Pete Cape, SSI, UK April Lecture Series 2014 Q & A Ray Poynter The Future Place Pete Cape SSI 2014 Platinum Sponsor April Series Sponsor Lecture Sponsor
Pete Cape, SSI, UK April Lecture Series 2014 | 186 | surveysampling.com info@surveysampling.com pete.cape@surveysampling.com

Recommandé

Wh.question
Wh.questionWh.question
Wh.questionmariam770
 
Preparing grade 3/ 4 Trinity June 2017
Preparing grade 3/ 4 Trinity June 2017Preparing grade 3/ 4 Trinity June 2017
Preparing grade 3/ 4 Trinity June 2017CEIP JOSÉ BERGAMÍN
 
Expressing Suggestion and Offer
Expressing Suggestion and OfferExpressing Suggestion and Offer
Expressing Suggestion and OfferIntan Sari Sadikin
 
Betsy leichliter april lecture series - 2014
Betsy leichliter april lecture series - 2014Betsy leichliter april lecture series - 2014
Betsy leichliter april lecture series - 2014Ray Poynter
 
Astrid And Sara Lost Items
Astrid And Sara Lost ItemsAstrid And Sara Lost Items
Astrid And Sara Lost ItemsJohn Dalesio
 
Interview laila1
Interview laila1Interview laila1
Interview laila1workventures
 
Writing to argue
Writing to argueWriting to argue
Writing to argueMrsCJ
 
Toefl ibt speaking
Toefl ibt speakingToefl ibt speaking
Toefl ibt speakingchudiozdi
 

Recommandé

Wh.question
Wh.questionWh.question
Wh.questionmariam770
 
Preparing grade 3/ 4 Trinity June 2017
Preparing grade 3/ 4 Trinity June 2017Preparing grade 3/ 4 Trinity June 2017
Preparing grade 3/ 4 Trinity June 2017CEIP JOSÉ BERGAMÍN
 
Expressing Suggestion and Offer
Expressing Suggestion and OfferExpressing Suggestion and Offer
Expressing Suggestion and OfferIntan Sari Sadikin
 
Betsy leichliter april lecture series - 2014
Betsy leichliter april lecture series - 2014Betsy leichliter april lecture series - 2014
Betsy leichliter april lecture series - 2014Ray Poynter
 
Astrid And Sara Lost Items
Astrid And Sara Lost ItemsAstrid And Sara Lost Items
Astrid And Sara Lost ItemsJohn Dalesio
 
Interview laila1
Interview laila1Interview laila1
Interview laila1workventures
 
Writing to argue
Writing to argueWriting to argue
Writing to argueMrsCJ
 
Toefl ibt speaking
Toefl ibt speakingToefl ibt speaking
Toefl ibt speakingchudiozdi
 
Ielts speaking part 1
Ielts speaking part 1Ielts speaking part 1
Ielts speaking part 1st0rmchazer
 
NNSTOY A Framework of Fun:Learning Through and From Games
NNSTOY A Framework of Fun:Learning Through and From Games NNSTOY A Framework of Fun:Learning Through and From Games
NNSTOY A Framework of Fun:Learning Through and From Games Curtis Chandler
 
Unclassroom elementary 2014
Unclassroom elementary 2014Unclassroom elementary 2014
Unclassroom elementary 2014Curtis Chandler
 
Names four corners intro
Names four corners intro Names four corners intro
Names four corners intro Fishkiller
 
Paired Choice
Paired ChoicePaired Choice
Paired Choiceebenimzo
 
Personal Preference
Personal PreferencePersonal Preference
Personal Preferenceebenimzo
 
Involve me and I'll understand: engaging students in research-led teaching
Involve me and I'll understand: engaging students in research-led teachingInvolve me and I'll understand: engaging students in research-led teaching
Involve me and I'll understand: engaging students in research-led teachingGary Wood
 
Suggest and Offer
Suggest and OfferSuggest and Offer
Suggest and OfferYudha Bhaskoro
 
Back To School Night Presentation 2009
Back To School Night Presentation 2009Back To School Night Presentation 2009
Back To School Night Presentation 2009jtiggs
 
Ielts Speaking Test1
Ielts Speaking Test1Ielts Speaking Test1
Ielts Speaking Test1ethanzhang
 
Top 10 discussion starters for children
Top 10 discussion starters for childrenTop 10 discussion starters for children
Top 10 discussion starters for childrenJackie Bolen
 
Technology and education
Technology and educationTechnology and education
Technology and educationmpaul01
 
Technology and education
Technology and educationTechnology and education
Technology and educationmpaul01
 
Minarets High PBE Sophomores 2014 Template - Matt Powers
Minarets High PBE Sophomores 2014 Template - Matt PowersMinarets High PBE Sophomores 2014 Template - Matt Powers
Minarets High PBE Sophomores 2014 Template - Matt PowersMatt Powers
 
Interviewing ppt
Interviewing pptInterviewing ppt
Interviewing pptjhiggins11
 
Peter cape training day - 2013
Peter cape training day - 2013Peter cape training day - 2013
Peter cape training day - 2013Ray Poynter
 
What I wish I knew in my first year of testing
What I wish I knew in my first year of testingWhat I wish I knew in my first year of testing
What I wish I knew in my first year of testing💻 Nicola Lindgren (née Owen)
 
Young researchers upload version
Young researchers upload versionYoung researchers upload version
Young researchers upload versionkatiecdec
 
Young Researchers
Young ResearchersYoung Researchers
Young Researcherskatiecdec
 
Improving Whole Class Inquiry Participation
Improving Whole Class Inquiry ParticipationImproving Whole Class Inquiry Participation
Improving Whole Class Inquiry ParticipationJoan Gallagher
 
The Art of Finding Your Story: NetSquared Vancouver 2013-08-14
The Art of Finding Your Story: NetSquared Vancouver 2013-08-14The Art of Finding Your Story: NetSquared Vancouver 2013-08-14
The Art of Finding Your Story: NetSquared Vancouver 2013-08-14NetSquared Vancouver
 
mark [Autosaved].ppt
mark [Autosaved].pptmark [Autosaved].ppt
mark [Autosaved].pptAndrewPruett3
 

Contenu connexe

Tendances

Ielts speaking part 1
Ielts speaking part 1Ielts speaking part 1
Ielts speaking part 1st0rmchazer
 
NNSTOY A Framework of Fun:Learning Through and From Games
NNSTOY A Framework of Fun:Learning Through and From Games NNSTOY A Framework of Fun:Learning Through and From Games
NNSTOY A Framework of Fun:Learning Through and From Games Curtis Chandler
 
Unclassroom elementary 2014
Unclassroom elementary 2014Unclassroom elementary 2014
Unclassroom elementary 2014Curtis Chandler
 
Names four corners intro
Names four corners intro Names four corners intro
Names four corners intro Fishkiller
 
Paired Choice
Paired ChoicePaired Choice
Paired Choiceebenimzo
 
Personal Preference
Personal PreferencePersonal Preference
Personal Preferenceebenimzo
 
Involve me and I'll understand: engaging students in research-led teaching
Involve me and I'll understand: engaging students in research-led teachingInvolve me and I'll understand: engaging students in research-led teaching
Involve me and I'll understand: engaging students in research-led teachingGary Wood
 
Back To School Night Presentation 2009
Back To School Night Presentation 2009Back To School Night Presentation 2009
Back To School Night Presentation 2009jtiggs
 
Ielts Speaking Test1
Ielts Speaking Test1Ielts Speaking Test1
Ielts Speaking Test1ethanzhang
 
Top 10 discussion starters for children
Top 10 discussion starters for childrenTop 10 discussion starters for children
Top 10 discussion starters for childrenJackie Bolen
 
Technology and education
Technology and educationTechnology and education
Technology and educationmpaul01
 
Technology and education
Technology and educationTechnology and education
Technology and educationmpaul01
 
Minarets High PBE Sophomores 2014 Template - Matt Powers
Minarets High PBE Sophomores 2014 Template - Matt PowersMinarets High PBE Sophomores 2014 Template - Matt Powers
Minarets High PBE Sophomores 2014 Template - Matt PowersMatt Powers
 

Tendances (14)

Ielts speaking part 1
Ielts speaking part 1Ielts speaking part 1
Ielts speaking part 1
 
NNSTOY A Framework of Fun:Learning Through and From Games
NNSTOY A Framework of Fun:Learning Through and From Games NNSTOY A Framework of Fun:Learning Through and From Games
NNSTOY A Framework of Fun:Learning Through and From Games
 
Unclassroom elementary 2014
Unclassroom elementary 2014Unclassroom elementary 2014
Unclassroom elementary 2014
 
Names four corners intro
Names four corners intro Names four corners intro
Names four corners intro
 
Paired Choice
Paired ChoicePaired Choice
Paired Choice
 
Personal Preference
Personal PreferencePersonal Preference
Personal Preference
 
Involve me and I'll understand: engaging students in research-led teaching
Involve me and I'll understand: engaging students in research-led teachingInvolve me and I'll understand: engaging students in research-led teaching
Involve me and I'll understand: engaging students in research-led teaching
 
Suggest and Offer
Suggest and OfferSuggest and Offer
Suggest and Offer
 
Back To School Night Presentation 2009
Back To School Night Presentation 2009Back To School Night Presentation 2009
Back To School Night Presentation 2009
 
Ielts Speaking Test1
Ielts Speaking Test1Ielts Speaking Test1
Ielts Speaking Test1
 
Top 10 discussion starters for children
Top 10 discussion starters for childrenTop 10 discussion starters for children
Top 10 discussion starters for children
 
Technology and education
Technology and educationTechnology and education
Technology and education
 
Technology and education
Technology and educationTechnology and education
Technology and education
 
Minarets High PBE Sophomores 2014 Template - Matt Powers
Minarets High PBE Sophomores 2014 Template - Matt PowersMinarets High PBE Sophomores 2014 Template - Matt Powers
Minarets High PBE Sophomores 2014 Template - Matt Powers
 

Similaire à Pete cape april lecture series - 2014

Interviewing ppt
Interviewing pptInterviewing ppt
Interviewing pptjhiggins11
 
Peter cape training day - 2013
Peter cape training day - 2013Peter cape training day - 2013
Peter cape training day - 2013Ray Poynter
 
Young researchers upload version
Young researchers upload versionYoung researchers upload version
Young researchers upload versionkatiecdec
 
Young Researchers
Young ResearchersYoung Researchers
Young Researcherskatiecdec
 
Improving Whole Class Inquiry Participation
Improving Whole Class Inquiry ParticipationImproving Whole Class Inquiry Participation
Improving Whole Class Inquiry ParticipationJoan Gallagher
 
The Art of Finding Your Story: NetSquared Vancouver 2013-08-14
The Art of Finding Your Story: NetSquared Vancouver 2013-08-14The Art of Finding Your Story: NetSquared Vancouver 2013-08-14
The Art of Finding Your Story: NetSquared Vancouver 2013-08-14NetSquared Vancouver
 
mark [Autosaved].ppt
mark [Autosaved].pptmark [Autosaved].ppt
mark [Autosaved].pptAndrewPruett3
 
Tips on Conducting Interviews
Tips on Conducting InterviewsTips on Conducting Interviews
Tips on Conducting InterviewsDavid Sheets
 
Interviewing101
Interviewing101Interviewing101
Interviewing101jhiggins11
 
Evaluation question 3
Evaluation question 3Evaluation question 3
Evaluation question 3bethan1234
 
Skills@work Presentation Skills Workshop - O'Connells - 5th March 2018
Skills@work Presentation Skills Workshop - O'Connells - 5th March 2018Skills@work Presentation Skills Workshop - O'Connells - 5th March 2018
Skills@work Presentation Skills Workshop - O'Connells - 5th March 2018Joe O'Donnell
 
Evaluation question 3
Evaluation question 3Evaluation question 3
Evaluation question 3bethan1234
 
Question Everything: The Power of Asking Questions
Question Everything: The Power of Asking QuestionsQuestion Everything: The Power of Asking Questions
Question Everything: The Power of Asking QuestionsDiane Windingland
 
Vsb an intro to afl.nov.2010
Vsb an intro to afl.nov.2010Vsb an intro to afl.nov.2010
Vsb an intro to afl.nov.2010Faye Brownlie
 
Talis Aspire User Group Australia Moving Forward
Talis Aspire User Group Australia Moving ForwardTalis Aspire User Group Australia Moving Forward
Talis Aspire User Group Australia Moving ForwardTalis
 
NC Paraprofessional 2019 - How To Ask Better Questions
NC Paraprofessional 2019 - How To Ask Better QuestionsNC Paraprofessional 2019 - How To Ask Better Questions
NC Paraprofessional 2019 - How To Ask Better QuestionsAmanda McLellan
 

Similaire à Pete cape april lecture series - 2014 (20)

Interviewing ppt
Interviewing pptInterviewing ppt
Interviewing ppt
 
Peter cape training day - 2013
Peter cape training day - 2013Peter cape training day - 2013
Peter cape training day - 2013
 
What I wish I knew in my first year of testing
What I wish I knew in my first year of testingWhat I wish I knew in my first year of testing
What I wish I knew in my first year of testing
 
Young researchers upload version
Young researchers upload versionYoung researchers upload version
Young researchers upload version
 
Young Researchers
Young ResearchersYoung Researchers
Young Researchers
 
Improving Whole Class Inquiry Participation
Improving Whole Class Inquiry ParticipationImproving Whole Class Inquiry Participation
Improving Whole Class Inquiry Participation
 
The Art of Finding Your Story: NetSquared Vancouver 2013-08-14
The Art of Finding Your Story: NetSquared Vancouver 2013-08-14The Art of Finding Your Story: NetSquared Vancouver 2013-08-14
The Art of Finding Your Story: NetSquared Vancouver 2013-08-14
 
mark [Autosaved].ppt
mark [Autosaved].pptmark [Autosaved].ppt
mark [Autosaved].ppt
 
Tips on Conducting Interviews
Tips on Conducting InterviewsTips on Conducting Interviews
Tips on Conducting Interviews
 
Interviewing101
Interviewing101Interviewing101
Interviewing101
 
Evaluation question 3
Evaluation question 3Evaluation question 3
Evaluation question 3
 
QA Analysis.pptx
QA Analysis.pptxQA Analysis.pptx
QA Analysis.pptx
 
Skills@work Presentation Skills Workshop - O'Connells - 5th March 2018
Skills@work Presentation Skills Workshop - O'Connells - 5th March 2018Skills@work Presentation Skills Workshop - O'Connells - 5th March 2018
Skills@work Presentation Skills Workshop - O'Connells - 5th March 2018
 
Evaluation question 3
Evaluation question 3Evaluation question 3
Evaluation question 3
 
Question Everything: The Power of Asking Questions
Question Everything: The Power of Asking QuestionsQuestion Everything: The Power of Asking Questions
Question Everything: The Power of Asking Questions
 
Vsb an intro to afl.nov.2010
Vsb an intro to afl.nov.2010Vsb an intro to afl.nov.2010
Vsb an intro to afl.nov.2010
 
mark2.ppt
mark2.pptmark2.ppt
mark2.ppt
 
Talis Aspire User Group Australia Moving Forward
Talis Aspire User Group Australia Moving ForwardTalis Aspire User Group Australia Moving Forward
Talis Aspire User Group Australia Moving Forward
 
NC Paraprofessional 2019 - How To Ask Better Questions
NC Paraprofessional 2019 - How To Ask Better QuestionsNC Paraprofessional 2019 - How To Ask Better Questions
NC Paraprofessional 2019 - How To Ask Better Questions
 
QualitativeLecture
QualitativeLectureQualitativeLecture
QualitativeLecture
 

Plus de Ray Poynter

The State of AI in Insights and Research 2024: Results and Findings
The State of AI in Insights and Research 2024: Results and FindingsThe State of AI in Insights and Research 2024: Results and Findings
The State of AI in Insights and Research 2024: Results and FindingsRay Poynter
 
ResearchWiseAI - an artificial intelligence driven research data analysis tool
ResearchWiseAI - an artificial intelligence driven research data analysis toolResearchWiseAI - an artificial intelligence driven research data analysis tool
ResearchWiseAI - an artificial intelligence driven research data analysis toolRay Poynter
 
AI-powered interviewing: Best practices from Yasna
AI-powered interviewing: Best practices from YasnaAI-powered interviewing: Best practices from Yasna
AI-powered interviewing: Best practices from YasnaRay Poynter
 
Artificial Intelligence and Qual: The Story So Far
Artificial Intelligence and Qual: The Story So FarArtificial Intelligence and Qual: The Story So Far
Artificial Intelligence and Qual: The Story So FarRay Poynter
 
State of Research Insights in Q1, 2024 from NewMR
State of Research Insights in Q1, 2024 from NewMRState of Research Insights in Q1, 2024 from NewMR
State of Research Insights in Q1, 2024 from NewMRRay Poynter
 
Sudden Death of Beliefs
Sudden Death of BeliefsSudden Death of Beliefs
Sudden Death of BeliefsRay Poynter
 
Uncovering Consumers’ Hidden Narratives
Uncovering Consumers’ Hidden NarrativesUncovering Consumers’ Hidden Narratives
Uncovering Consumers’ Hidden NarrativesRay Poynter
 
Narrative Exploration of New Categories at Mondelēz
Narrative Exploration of New Categories at MondelēzNarrative Exploration of New Categories at Mondelēz
Narrative Exploration of New Categories at MondelēzRay Poynter
 
The Future in Focus
The Future in FocusThe Future in Focus
The Future in FocusRay Poynter
 
The Future in Focus
The Future in FocusThe Future in Focus
The Future in FocusRay Poynter
 
The State of Insights – September 2023
The State of Insights – September 2023The State of Insights – September 2023
The State of Insights – September 2023Ray Poynter
 
Research Thinking in the age of AI
Research Thinking in the age of AIResearch Thinking in the age of AI
Research Thinking in the age of AIRay Poynter
 
How might AI impact Research and Insights over the next two years?
How might AI impact Research and Insights over the next two years?How might AI impact Research and Insights over the next two years?
How might AI impact Research and Insights over the next two years?Ray Poynter
 
From Words to Wisdom: Unleashing the Potential of Language Models for Human-C...
From Words to Wisdom: Unleashing the Potential of Language Models for Human-C...From Words to Wisdom: Unleashing the Potential of Language Models for Human-C...
From Words to Wisdom: Unleashing the Potential of Language Models for Human-C...Ray Poynter
 
ChatGPT for Social Media Listening: practical application with YouScan’s Insi...
ChatGPT for Social Media Listening: practical application with YouScan’s Insi...ChatGPT for Social Media Listening: practical application with YouScan’s Insi...
ChatGPT for Social Media Listening: practical application with YouScan’s Insi...Ray Poynter
 
Using Generative AI to Assess the Quality of Open-Ended Responses in Surveys
Using Generative AI to Assess the Quality of Open-Ended Responses in SurveysUsing Generative AI to Assess the Quality of Open-Ended Responses in Surveys
Using Generative AI to Assess the Quality of Open-Ended Responses in SurveysRay Poynter
 
Exploring the future of verbatim coding with ChatGPT
Exploring the future of verbatim coding with ChatGPTExploring the future of verbatim coding with ChatGPT
Exploring the future of verbatim coding with ChatGPTRay Poynter
 
Using Generative AI to bring Qualitative Capabilities to Quantitative Surveys
Using Generative AI to bring Qualitative Capabilities to Quantitative SurveysUsing Generative AI to bring Qualitative Capabilities to Quantitative Surveys
Using Generative AI to bring Qualitative Capabilities to Quantitative SurveysRay Poynter
 
How AI / ChatGPT Drives Business Growth
How AI / ChatGPT Drives Business GrowthHow AI / ChatGPT Drives Business Growth
How AI / ChatGPT Drives Business GrowthRay Poynter
 
Tech for tech’s sake? Learnings from experiments with AI in consumer research
Tech for tech’s sake? Learnings from experiments with AI in consumer researchTech for tech’s sake? Learnings from experiments with AI in consumer research
Tech for tech’s sake? Learnings from experiments with AI in consumer researchRay Poynter
 

Plus de Ray Poynter (20)

The State of AI in Insights and Research 2024: Results and Findings
The State of AI in Insights and Research 2024: Results and FindingsThe State of AI in Insights and Research 2024: Results and Findings
The State of AI in Insights and Research 2024: Results and Findings
 
ResearchWiseAI - an artificial intelligence driven research data analysis tool
ResearchWiseAI - an artificial intelligence driven research data analysis toolResearchWiseAI - an artificial intelligence driven research data analysis tool
ResearchWiseAI - an artificial intelligence driven research data analysis tool
 
AI-powered interviewing: Best practices from Yasna
AI-powered interviewing: Best practices from YasnaAI-powered interviewing: Best practices from Yasna
AI-powered interviewing: Best practices from Yasna
 
Artificial Intelligence and Qual: The Story So Far
Artificial Intelligence and Qual: The Story So FarArtificial Intelligence and Qual: The Story So Far
Artificial Intelligence and Qual: The Story So Far
 
State of Research Insights in Q1, 2024 from NewMR
State of Research Insights in Q1, 2024 from NewMRState of Research Insights in Q1, 2024 from NewMR
State of Research Insights in Q1, 2024 from NewMR
 
Sudden Death of Beliefs
Sudden Death of BeliefsSudden Death of Beliefs
Sudden Death of Beliefs
 
Uncovering Consumers’ Hidden Narratives
Uncovering Consumers’ Hidden NarrativesUncovering Consumers’ Hidden Narratives
Uncovering Consumers’ Hidden Narratives
 
Narrative Exploration of New Categories at Mondelēz
Narrative Exploration of New Categories at MondelēzNarrative Exploration of New Categories at Mondelēz
Narrative Exploration of New Categories at Mondelēz
 
The Future in Focus
The Future in FocusThe Future in Focus
The Future in Focus
 
The Future in Focus
The Future in FocusThe Future in Focus
The Future in Focus
 
The State of Insights – September 2023
The State of Insights – September 2023The State of Insights – September 2023
The State of Insights – September 2023
 
Research Thinking in the age of AI
Research Thinking in the age of AIResearch Thinking in the age of AI
Research Thinking in the age of AI
 
How might AI impact Research and Insights over the next two years?
How might AI impact Research and Insights over the next two years?How might AI impact Research and Insights over the next two years?
How might AI impact Research and Insights over the next two years?
 
From Words to Wisdom: Unleashing the Potential of Language Models for Human-C...
From Words to Wisdom: Unleashing the Potential of Language Models for Human-C...From Words to Wisdom: Unleashing the Potential of Language Models for Human-C...
From Words to Wisdom: Unleashing the Potential of Language Models for Human-C...
 
ChatGPT for Social Media Listening: practical application with YouScan’s Insi...
ChatGPT for Social Media Listening: practical application with YouScan’s Insi...ChatGPT for Social Media Listening: practical application with YouScan’s Insi...
ChatGPT for Social Media Listening: practical application with YouScan’s Insi...
 
Using Generative AI to Assess the Quality of Open-Ended Responses in Surveys
Using Generative AI to Assess the Quality of Open-Ended Responses in SurveysUsing Generative AI to Assess the Quality of Open-Ended Responses in Surveys
Using Generative AI to Assess the Quality of Open-Ended Responses in Surveys
 
Exploring the future of verbatim coding with ChatGPT
Exploring the future of verbatim coding with ChatGPTExploring the future of verbatim coding with ChatGPT
Exploring the future of verbatim coding with ChatGPT
 
Using Generative AI to bring Qualitative Capabilities to Quantitative Surveys
Using Generative AI to bring Qualitative Capabilities to Quantitative SurveysUsing Generative AI to bring Qualitative Capabilities to Quantitative Surveys
Using Generative AI to bring Qualitative Capabilities to Quantitative Surveys
 
How AI / ChatGPT Drives Business Growth
How AI / ChatGPT Drives Business GrowthHow AI / ChatGPT Drives Business Growth
How AI / ChatGPT Drives Business Growth
 
Tech for tech’s sake? Learnings from experiments with AI in consumer research
Tech for tech’s sake? Learnings from experiments with AI in consumer researchTech for tech’s sake? Learnings from experiments with AI in consumer research
Tech for tech’s sake? Learnings from experiments with AI in consumer research
 

Dernier

DGR_Digital Advertising Strategies for a Cookieless World_Presentation.pdf
DGR_Digital Advertising Strategies for a Cookieless World_Presentation.pdfDGR_Digital Advertising Strategies for a Cookieless World_Presentation.pdf
DGR_Digital Advertising Strategies for a Cookieless World_Presentation.pdfDemandbase
 
What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...
What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...
What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...Ahrefs
 
Digital Marketing Spotlight: Lifecycle Advertising Strategies.pdf
Digital Marketing Spotlight: Lifecycle Advertising Strategies.pdfDigital Marketing Spotlight: Lifecycle Advertising Strategies.pdf
Digital Marketing Spotlight: Lifecycle Advertising Strategies.pdfDemandbase
 
Master the Art of Digital Recruitment in Asia.pdf
Master the Art of Digital Recruitment in Asia.pdfMaster the Art of Digital Recruitment in Asia.pdf
Master the Art of Digital Recruitment in Asia.pdfHigher Education Marketing
 
How To Utilize Calculated Properties in your HubSpot Setup
How To Utilize Calculated Properties in your HubSpot SetupHow To Utilize Calculated Properties in your HubSpot Setup
How To Utilize Calculated Properties in your HubSpot Setupssuser4571da
 
Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...
Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...
Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...CIO Business World
 
TAM AdEx 2023 Cross Media Advertising Recap - Auto Sector
TAM AdEx 2023 Cross Media Advertising Recap - Auto SectorTAM AdEx 2023 Cross Media Advertising Recap - Auto Sector
TAM AdEx 2023 Cross Media Advertising Recap - Auto SectorSocial Samosa
 
Call Girls In Aerocity Delhi ❤️8860477959 Good Looking Escorts In 24/7 Delhi NCR
Call Girls In Aerocity Delhi ❤️8860477959 Good Looking Escorts In 24/7 Delhi NCRCall Girls In Aerocity Delhi ❤️8860477959 Good Looking Escorts In 24/7 Delhi NCR
Call Girls In Aerocity Delhi ❤️8860477959 Good Looking Escorts In 24/7 Delhi NCRlizamodels9
 
From Chance to Choice - Tactical Link Building for International SEO
From Chance to Choice - Tactical Link Building for International SEOFrom Chance to Choice - Tactical Link Building for International SEO
From Chance to Choice - Tactical Link Building for International SEOSzymon Słowik
 
Storyboards for my Final Major Project Video
Storyboards for my Final Major Project VideoStoryboards for my Final Major Project Video
Storyboards for my Final Major Project VideoSineadBidwell
 
Talent Management for mba 3rd sem useful
Talent Management for mba 3rd sem usefulTalent Management for mba 3rd sem useful
Talent Management for mba 3rd sem usefulAtifaArbar
 
2024's Top PPC Tactics: Triple Your Google Ads Local Leads
2024's Top PPC Tactics: Triple Your Google Ads Local Leads2024's Top PPC Tactics: Triple Your Google Ads Local Leads
2024's Top PPC Tactics: Triple Your Google Ads Local LeadsSearch Engine Journal
 
Call Girls in Lajpat Nagar Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Lajpat Nagar Delhi 💯Call Us 🔝8264348440🔝Call Girls in Lajpat Nagar Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Lajpat Nagar Delhi 💯Call Us 🔝8264348440🔝soniya singh
 
The Pitfalls of Keyword Stuffing in SEO Copywriting
The Pitfalls of Keyword Stuffing in SEO CopywritingThe Pitfalls of Keyword Stuffing in SEO Copywriting
The Pitfalls of Keyword Stuffing in SEO CopywritingJuan Pineda
 
marketing strategy of tanishq word PPROJECT.pdf
marketing strategy of tanishq word PPROJECT.pdfmarketing strategy of tanishq word PPROJECT.pdf
marketing strategy of tanishq word PPROJECT.pdfarsathsahil
 
Mastering SEO in the Evolving AI-driven World
Mastering SEO in the Evolving AI-driven WorldMastering SEO in the Evolving AI-driven World
Mastering SEO in the Evolving AI-driven WorldScalenut
 
定制(ULV毕业证书)拉文大学毕业证成绩单原版一比一
定制(ULV毕业证书)拉文大学毕业证成绩单原版一比一定制(ULV毕业证书)拉文大学毕业证成绩单原版一比一
定制(ULV毕业证书)拉文大学毕业证成绩单原版一比一s SS
 
McDonald's: A Journey Through Time (PPT)
McDonald's: A Journey Through Time (PPT)McDonald's: A Journey Through Time (PPT)
McDonald's: A Journey Through Time (PPT)DEVARAJV16
 
2024 SEO Trends for Business Success (WSA)
2024 SEO Trends for Business Success (WSA)2024 SEO Trends for Business Success (WSA)
2024 SEO Trends for Business Success (WSA)Jomer Gregorio
 
VIP Call Girls In Green Park 9654467111 Escorts Service
VIP Call Girls In Green Park 9654467111 Escorts ServiceVIP Call Girls In Green Park 9654467111 Escorts Service
VIP Call Girls In Green Park 9654467111 Escorts ServiceSapana Sha
 

Dernier (20)

DGR_Digital Advertising Strategies for a Cookieless World_Presentation.pdf
DGR_Digital Advertising Strategies for a Cookieless World_Presentation.pdfDGR_Digital Advertising Strategies for a Cookieless World_Presentation.pdf
DGR_Digital Advertising Strategies for a Cookieless World_Presentation.pdf
 
What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...
What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...
What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...
 
Digital Marketing Spotlight: Lifecycle Advertising Strategies.pdf
Digital Marketing Spotlight: Lifecycle Advertising Strategies.pdfDigital Marketing Spotlight: Lifecycle Advertising Strategies.pdf
Digital Marketing Spotlight: Lifecycle Advertising Strategies.pdf
 
Master the Art of Digital Recruitment in Asia.pdf
Master the Art of Digital Recruitment in Asia.pdfMaster the Art of Digital Recruitment in Asia.pdf
Master the Art of Digital Recruitment in Asia.pdf
 
How To Utilize Calculated Properties in your HubSpot Setup
How To Utilize Calculated Properties in your HubSpot SetupHow To Utilize Calculated Properties in your HubSpot Setup
How To Utilize Calculated Properties in your HubSpot Setup
 
Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...
Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...
Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...
 
TAM AdEx 2023 Cross Media Advertising Recap - Auto Sector
TAM AdEx 2023 Cross Media Advertising Recap - Auto SectorTAM AdEx 2023 Cross Media Advertising Recap - Auto Sector
TAM AdEx 2023 Cross Media Advertising Recap - Auto Sector
 
Call Girls In Aerocity Delhi ❤️8860477959 Good Looking Escorts In 24/7 Delhi NCR
Call Girls In Aerocity Delhi ❤️8860477959 Good Looking Escorts In 24/7 Delhi NCRCall Girls In Aerocity Delhi ❤️8860477959 Good Looking Escorts In 24/7 Delhi NCR
Call Girls In Aerocity Delhi ❤️8860477959 Good Looking Escorts In 24/7 Delhi NCR
 
From Chance to Choice - Tactical Link Building for International SEO
From Chance to Choice - Tactical Link Building for International SEOFrom Chance to Choice - Tactical Link Building for International SEO
From Chance to Choice - Tactical Link Building for International SEO
 
Storyboards for my Final Major Project Video
Storyboards for my Final Major Project VideoStoryboards for my Final Major Project Video
Storyboards for my Final Major Project Video
 
Talent Management for mba 3rd sem useful
Talent Management for mba 3rd sem usefulTalent Management for mba 3rd sem useful
Talent Management for mba 3rd sem useful
 
2024's Top PPC Tactics: Triple Your Google Ads Local Leads
2024's Top PPC Tactics: Triple Your Google Ads Local Leads2024's Top PPC Tactics: Triple Your Google Ads Local Leads
2024's Top PPC Tactics: Triple Your Google Ads Local Leads
 
Call Girls in Lajpat Nagar Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Lajpat Nagar Delhi 💯Call Us 🔝8264348440🔝Call Girls in Lajpat Nagar Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Lajpat Nagar Delhi 💯Call Us 🔝8264348440🔝
 
The Pitfalls of Keyword Stuffing in SEO Copywriting
The Pitfalls of Keyword Stuffing in SEO CopywritingThe Pitfalls of Keyword Stuffing in SEO Copywriting
The Pitfalls of Keyword Stuffing in SEO Copywriting
 
marketing strategy of tanishq word PPROJECT.pdf
marketing strategy of tanishq word PPROJECT.pdfmarketing strategy of tanishq word PPROJECT.pdf
marketing strategy of tanishq word PPROJECT.pdf
 
Mastering SEO in the Evolving AI-driven World
Mastering SEO in the Evolving AI-driven WorldMastering SEO in the Evolving AI-driven World
Mastering SEO in the Evolving AI-driven World
 
定制(ULV毕业证书)拉文大学毕业证成绩单原版一比一
定制(ULV毕业证书)拉文大学毕业证成绩单原版一比一定制(ULV毕业证书)拉文大学毕业证成绩单原版一比一
定制(ULV毕业证书)拉文大学毕业证成绩单原版一比一
 
McDonald's: A Journey Through Time (PPT)
McDonald's: A Journey Through Time (PPT)McDonald's: A Journey Through Time (PPT)
McDonald's: A Journey Through Time (PPT)
 
2024 SEO Trends for Business Success (WSA)
2024 SEO Trends for Business Success (WSA)2024 SEO Trends for Business Success (WSA)
2024 SEO Trends for Business Success (WSA)
 
VIP Call Girls In Green Park 9654467111 Escorts Service
VIP Call Girls In Green Park 9654467111 Escorts ServiceVIP Call Girls In Green Park 9654467111 Escorts Service
VIP Call Girls In Green Park 9654467111 Escorts Service
 

Pete cape april lecture series - 2014

  • 1. Pete Cape, SSI, UK April Lecture Series 2014 Questionnaire Design – A Masterclass Pete Cape Global Knowledge Director, SSI 2014 Platinum Sponsor April Series Sponsor Lecture Sponsor
  • 2. Pete Cape, SSI, UK April Lecture Series 2014 | 2 | A questionnaire might have all of these…. • Open questions • Yes/No questions • Single coded questions with multiple answers • Multi-coded questions with multiple answers • Items to be ranked • Items to be rated • Items to be chosen between
  • 3. Pete Cape, SSI, UK April Lecture Series 2014 | 3 | A questionnaire might have all of these…. • Open questions • Yes/No questions • Single coded questions with multiple answers • Multi-coded questions with multiple answers • Items to be ranked • Items to be rated • Items to be chosen between • But this is such a boring way of looking at it….
  • 4. Pete Cape, SSI, UK April Lecture Series 2014 | 4 | A questionnaire might have all of these…. • Open questions • Yes/No questions • Single coded questions with multiple answers • Multi-coded questions with multiple answers • Items to be ranked • Items to be rated • Items to be chosen between • But this is such a boring way of looking at it…. • Let’s start with some principles….
  • 5. Pete Cape, SSI, UK April Lecture Series 2014 | 5 | Things to remember about respondents….
  • 6. Pete Cape, SSI, UK April Lecture Series 2014 | 6 | Things to remember about respondents…. • They know they are in an experiment
  • 7. Pete Cape, SSI, UK April Lecture Series 2014 | 7 | Things to remember about respondents…. • They know they are in an experiment • Everything they see, hear or read is meant deliberately, and has meaning
  • 8. Pete Cape, SSI, UK April Lecture Series 2014 | 8 | Things to remember about respondents…. • They know they are in an experiment • Everything they see, hear or read is meant deliberately, and has meaning • They want to please
  • 9. Pete Cape, SSI, UK April Lecture Series 2014 | 9 | Things to remember about respondents…. • They know they are in an experiment • Everything they see, hear or read is meant deliberately, and has meaning • They want to please • They think you know what you are doing…
  • 10. Pete Cape, SSI, UK April Lecture Series 2014
  • 11. Pete Cape, SSI, UK April Lecture Series 2014
  • 12. Pete Cape, SSI, UK April Lecture Series 2014 | 12 | Things to remember about questionnaires…. • Everything has meaning
  • 13. Pete Cape, SSI, UK April Lecture Series 2014 | 13 | Things to remember about questionnaires…. • Everything has meaning • You are supposed to know what that meaning is
  • 14. Pete Cape, SSI, UK April Lecture Series 2014 | 14 | Questionnaires do a lot of work • Motivate the respondent
  • 15. Pete Cape, SSI, UK April Lecture Series 2014 | 15 | Questionnaires do a lot of work • Motivate the respondent to provide — complete answers — accurate answers
  • 16. Pete Cape, SSI, UK April Lecture Series 2014 | 16 | Questionnaires do a lot of work • Motivate the respondent to provide — complete answers — accurate answers • Communicate to the respondent
  • 17. Pete Cape, SSI, UK April Lecture Series 2014 | 17 | Questionnaires do a lot of work • Motivate the respondent to provide — complete answers — accurate answers • Communicate to the respondent • Help the respondent work out their answers
  • 18. Pete Cape, SSI, UK April Lecture Series 2014 | 18 | Questionnaires do a lot of work • Motivate the respondent to provide — complete answers — accurate answers • Communicate to the respondent • Help the respondent work out their answers • Make the respondents task easy
  • 19. Pete Cape, SSI, UK April Lecture Series 2014 | 19 | Is this the simplest question of all? • Yes • No
  • 20. Pete Cape, SSI, UK April Lecture Series 2014 | 20 | Respondents want to please Do you care to disagree with me….? I am a scientist….. • Acquiescence bias
  • 21. Pete Cape, SSI, UK April Lecture Series 2014 Acquiescence bias • Some respondents are simply agreeable, and indicate agreement out of politeness. Saris, Krosnick and Shaeffe
  • 22. Pete Cape, SSI, UK April Lecture Series 2014 Acquiescence bias • Some respondents are simply agreeable, and indicate agreement out of politeness. • Other respondents expect that the researchers agree with the listed items and defer to their judgment. Saris, Krosnick and Shaeffe
  • 23. Pete Cape, SSI, UK April Lecture Series 2014 Acquiescence bias • Some respondents are simply agreeable, and indicate agreement out of politeness. • Other respondents expect that the researchers agree with the listed items and defer to their judgment. • Most respondents engage in survey satisficing and find that agreeing takes less effort than carefully weighing each optional level of disagreement and agreement. Saris, Krosnick and Shaeffe
  • 24. Pete Cape, SSI, UK April Lecture Series 2014 Affirmation vs Multicode
  • 25. Pete Cape, SSI, UK April Lecture Series 2014 Affirmation vs Multicode
  • 26. Pete Cape, SSI, UK April Lecture Series 2014 Overcoming acquiescence bias • Find them through traps and exclude
  • 27. Pete Cape, SSI, UK April Lecture Series 2014 Overcoming acquiescence bias • Find them through traps and exclude • Follow up question “how sure?” • If not 100% sure then recode as “no”
  • 28. Pete Cape, SSI, UK April Lecture Series 2014 Overcoming acquiescence bias • Find them through traps and exclude • Follow up question “how sure?” • If not 100% sure then recode as “no” • Avoid the question type and give alternatives — “chose the one of these that best describes you” — “to what extent do you…”
  • 29. Pete Cape, SSI, UK April Lecture Series 2014 Affirmation vs Multicode
  • 30. Pete Cape, SSI, UK April Lecture Series 2014 Affirmation vs Multicode why so big…?
  • 31. Pete Cape, SSI, UK April Lecture Series 2014 Social Desirability bias • Wanting to look good
  • 32. Pete Cape, SSI, UK April Lecture Series 2014 Social Desirability bias • Wanting to look good — To society — To other individuals — To myself
  • 33. Pete Cape, SSI, UK April Lecture Series 2014 Some Normal Everyday Things…that I keep to myself.. Chart Title
  • 34. Pete Cape, SSI, UK April Lecture Series 2014 Some Normal Everyday Things…I want you to think I do.. Chart Title
  • 35. Pete Cape, SSI, UK April Lecture Series 2014 20% bought a book last week? | 35 |
  • 36. Pete Cape, SSI, UK April Lecture Series 2014 Avoiding Social Desirability • Is very hard • Try to make respondents feel “normal”
  • 37. Pete Cape, SSI, UK April Lecture Series 2014 Avoiding Social Desirability • Is very hard • Try to make respondents feel “normal” This third section is about social attitudes and values. There is a lot of talk about discrimination and racial prejudice, and different people have different views on the size of such problems, their causes and what should be done about them. There are no right or wrong answers, it is your honest opinions we are looking for. If you do not wish to answer any of these questions please use the option “prefer not to say”.
  • 38. Pete Cape, SSI, UK April Lecture Series 2014 Questionnaire Design and Bias • Questionnaire design has to overcome respondents’ psychological biases
  • 39. Pete Cape, SSI, UK April Lecture Series 2014 Questionnaire Design and Bias • Questionnaire design has to overcome respondents’ psychological biases • And also experimenter bias
  • 40. Pete Cape, SSI, UK April Lecture Series 2014 Questionnaire Design and Bias • Questionnaire design has to overcome respondents’ psychological biases • And also experimenter bias • i.e. you and your client….
  • 41. Pete Cape, SSI, UK April Lecture Series 2014 Experimenter biases • It is easy to suggest the answer…
  • 42. Pete Cape, SSI, UK April Lecture Series 2014 Question suggests the answer Which carbonated soft drinks, like Coca-Cola or Pepsi for example, do you drink regularly?”
  • 43. Pete Cape, SSI, UK April Lecture Series 2014 Question suggests the answer Which carbonated soft drinks, like Coca-Cola or Pepsi for example, do you drink regularly?” Raises answers for drinks “like” Pepsi and Coke i.e. Brown or cola drinks
  • 44. Pete Cape, SSI, UK April Lecture Series 2014 Question suggests the answer Which carbonated soft drinks, like Coca-Cola or Pepsi for example, do you drink regularly?” Raises answers for drinks “like” Pepsi and Coke i.e. Brown or cola drinks Lowers answers for non- brown drinks e.g. Fanta, 7-Up
  • 45. Pete Cape, SSI, UK April Lecture Series 2014 Question suggests the answer Which carbonated soft drinks, like Coca-Cola or Pepsi for example, do you drink regularly?” Raises answers for drinks “like” Pepsi and Coke i.e. Brown or cola drinks Lowers answers for non- brown drinks e.g. Fanta, 7-Up Problem is in use of jargon term: “carbonated soft drinks”
  • 46. Pete Cape, SSI, UK April Lecture Series 2014 Question suggests “my” answer Testing perfumes on animals involves causing them pain. Do you agree with testing perfumes on animals?
  • 47. Pete Cape, SSI, UK April Lecture Series 2014 Question suggests “my” answer Testing perfumes on animals involves causing them pain. Do you agree with testing perfumes on animals? “involves causing them pain” is an opinion
  • 48. Pete Cape, SSI, UK April Lecture Series 2014 Avoiding Experimenter Bias • Is easier
  • 49. Pete Cape, SSI, UK April Lecture Series 2014 Avoiding Experimenter Bias • Is easier • Think neutral
  • 50. Pete Cape, SSI, UK April Lecture Series 2014 Avoiding Experimenter Bias • Is easier • Think neutral • Don’t think your view is the world view
  • 51. Pete Cape, SSI, UK April Lecture Series 2014 Questionnaire as a measuring tool • We rarely measure physical things
  • 52. Pete Cape, SSI, UK April Lecture Series 2014 Questionnaire as a measuring tool • We rarely measure physical things • But we do measure frequency: — How often do you x, y, z?
  • 53. Pete Cape, SSI, UK April Lecture Series 2014 Questionnaire as a measuring tool • We rarely measure physical things • But we do measure frequency: — How often do you x, y, z? • And we do measure attitudes: — How much do you think x,y,z?
  • 54. Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • As an relative concept in a fixed time period
  • 55. Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • As a relative concept in a fixed time period • Define: — The activity • What does it mean to “go shopping”? • What does it mean to “surf the internet?” — The time period • Is it the relevant one? • What is an “average week”? — The measurement scale • Always, often, sometimes, never…..
  • 56. Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • As an absolute number in a fixed time period
  • 57. Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • As an absolute number in a fixed time period • Define: — The activity • What does it mean to “go shopping”? • What does it mean to “surf the internet?” — The time period • Is it the relevant one? • What is an “average week”?
  • 58. Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • As an absolute number in a fixed time period How many times did you take a bath in the past year?
  • 59. Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • As an absolute number in a fixed time period How many times did you take a bath in the past year? too longwhat does this mean?
  • 60. Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • Precision and correct framing How many times, if at all, did you take a bath in the past week? By taking a bath we mean actually using a bath tub and filling it with water and sitting in it. Please exclude any showers you may have taken
  • 61. Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • Precision and correct framing How many times, if at all, did you take a bath in the past week? By taking a bath we mean actually using a bath tub and filling it with water and sitting in it. Please exclude any showers you may have taken OMG, I only take showers, he will think I am so dirty
  • 62. Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • Precision and correct framing, and overcoming social desirability bias Some people only take baths, some only take showers, some take both. This part of the survey is only concerned with bath taking. How many times, if at all, did you take a bath in the past week? By taking a bath we mean actually using a bath tub and filling it with water and sitting in it. Please exclude any showers you may have taken
  • 63. Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • Precision and correct framing, and overcoming social desirability bias, and to make life easy Some people only take baths, some only take showers, some take both. This part of the survey is only concerned with bath taking. How many times, if at all, did you take a bath in the past week? By taking a bath we mean actually using a bath tub and filling it with water and sitting in it. Please exclude any showers you may have taken 0 / only took showers 1 – 2 3 – 4 5 – 6 7 or more Can’t remember
  • 64. Pete Cape, SSI, UK April Lecture Series 2014 Measuring frequency • Precision and correct framing, and overcoming social desirability bias, and to make life easy Some people only take baths, some only take showers, some take both. This part of the survey is only concerned with bath taking. How many times, if at all, did you take a bath in the past week? By taking a bath we mean actually using a bath tub and filling it with water and sitting in it. Please exclude any showers you may have taken 0 / only took showers 1 – 2 3 – 4 5 – 6 7 or more Can’t remember Does this have any meaning?
  • 65. Pete Cape, SSI, UK April Lecture Series 2014 The meaning of ranges
  • 66. Pete Cape, SSI, UK April Lecture Series 2014 The meaning of ranges • Assumption of normal distribution
  • 67. Pete Cape, SSI, UK April Lecture Series 2014 The meaning of ranges • Assumption of normal distribution
  • 68. Pete Cape, SSI, UK April Lecture Series 2014 Manipulating numbers Answer list A Answer list B
  • 69. Pete Cape, SSI, UK April Lecture Series 2014 Manipulating numbers Answer list A Answer list B
  • 70. Pete Cape, SSI, UK April Lecture Series 2014 Manipulating numbers Answer list A Answer list B
  • 71. Pete Cape, SSI, UK April Lecture Series 2014 Manipulating numbers Answer list A • The central banding should contain the mean/median • But we don’t know it, it’s what we’re trying to find out! Answer list B
  • 72. Pete Cape, SSI, UK April Lecture Series 2014 Collecting real numbers • Just ask for it • Respondent knows it and will give it
  • 73. Pete Cape, SSI, UK April Lecture Series 2014 Measuring attitudes
  • 74. Pete Cape, SSI, UK April Lecture Series 2014 Measuring attitudes Fill in the words that correspond to the scale points for “Likely” 0 = not at all likely 1 = ? 2 = ? 3 = ? 4 = ? 5 = ? 6 = ? 7 = ? 8 = ? 9 = ? 10 = extremely likely ? ? Exercise
  • 75. Pete Cape, SSI, UK April Lecture Series 2014 Measuring attitudes Fill in the words that correspond to the scale points for “Likely” 0 = not at all likely 1 = ? 2 = ? 3 = ? 4 = ? 5 = ? 6 = ? 7 = ? 8 = ? 9 = ? 10 = extremely likely Exercise
  • 76. Pete Cape, SSI, UK April Lecture Series 2014 Measuring attitudes Fill in the words that correspond to the scale points for “Likely” 0 = not at all likely 1 = ? 2 = ? 3 = ? 4 = ? 5 = ? 6 = ? 7 = ? 8 = ? 9 = ? 10 = extremely likely Exercise Exercise outcomes: a) No-one will be able to do it b) Some people will write: 5 = “neither likely not unlikely” 5 = “some degree of likelihood”
  • 77. Pete Cape, SSI, UK April Lecture Series 2014 Measuring attitudes Fill in the words that correspond to the scale points for “Likely” 0 = not at all likely 1 = ? 2 = ? 3 = ? 4 = ? 5 = ? 6 = ? 7 = ? 8 = ? 9 = ? 10 = extremely likely Exercise Exercise outcomes: a) No-one will be able to do it b) Some people will write: 5 = “neither likely nor unlikely” 5 = “some degree of likelihood”
  • 78. Pete Cape, SSI, UK April Lecture Series 2014 Two men go into a bar….
  • 79. Pete Cape, SSI, UK April Lecture Series 2014 Two men go into a bar…. I say, what do you think of the new VW Golf?
  • 80. Pete Cape, SSI, UK April Lecture Series 2014 Two men go into a bar…. I say, what do you think of the new VW Golf? I think it’s a 6…
  • 81. Pete Cape, SSI, UK April Lecture Series 2014 Two men go into a bar…. Attitudes are expressed in words not numbers I say, what do you think of the new VW Golf? I think it’s a 6…
  • 82. Pete Cape, SSI, UK April Lecture Series 2014 Numbers are universal
  • 83. Pete Cape, SSI, UK April Lecture Series 2014 Numbers are universal • Numeric scales are not!
  • 84. Pete Cape, SSI, UK April Lecture Series 2014 Numbers are universal • Numeric scales are not! • It all starts in school — 0 = you weren’t even there! — 1-4 = you might as well not have been there… — 5-6 = you are going to have to work way harder in the future — 7-8 = okay, could do better — 9 = good — 10 = very good
  • 85. Pete Cape, SSI, UK April Lecture Series 2014 Numbers are universal • Numeric scales are not! • It all starts in school — 0 = you weren’t even there! — 1-4 = you might as well not have been there… — 5-6 = you are going to have to work way harder in the future — 7-8 = okay, could do better — 9 = good — 10 = very good • And this is for cultures that score 0 - 10, does everyone?
  • 86. Pete Cape, SSI, UK April Lecture Series 2014 German School System • 1 (sehr gut, very good) is the best possible grade and is given for outstanding performance • 2 (gut, good) is the next-highest and is given for performance that meets the standard completely and is above-average • 3 (befriedigend, satisfactory) indicates "average" performance. • 4 (ausreichend, sufficient) is the lowest passing grade and is given if the standard has been met but with a number of notable errors. • 5 (mangelhaft, deficient) is the higher of two failing grades and is given if the standard has not been met but the basics have been understood. • 6 (ungenügend, insufficient) is the lowest possible grade and is given if the standard has not been met and the basics have not been understood.
  • 87. Pete Cape, SSI, UK April Lecture Series 2014 Numbers have meaning 0 1 2 3 4 5 6 7 8 9 10 Schwartz, 1996
  • 88. Pete Cape, SSI, UK April Lecture Series 2014 Numbers have meaning 0 1 2 3 4 5 6 7 8 9 10 34% Schwartz, 1996
  • 89. Pete Cape, SSI, UK April Lecture Series 2014 Numbers have meaning 0 1 2 3 4 5 6 7 8 9 10 -5 -4 -3 -2 -1 0 1 2 3 4 5 34% Schwartz, 1996
  • 90. Pete Cape, SSI, UK April Lecture Series 2014 Numbers have meaning 0 1 2 3 4 5 6 7 8 9 10 -5 -4 -3 -2 -1 0 1 2 3 4 5 34% 13%? Schwartz, 1996
  • 91. Pete Cape, SSI, UK April Lecture Series 2014 Numbers have meaning 0 1 2 3 4 5 6 7 8 9 10 -5 -4 -3 -2 -1 0 1 2 3 4 5 34% 13% Schwartz, 1996
  • 92. Pete Cape, SSI, UK April Lecture Series 2014 This what scales are trying to achieve
  • 93. Pete Cape, SSI, UK April Lecture Series 2014 This what scales are trying to achieve
  • 94. Pete Cape, SSI, UK April Lecture Series 2014 • Turn to academia for the answer How long should a scale be?
  • 95. Pete Cape, SSI, UK April Lecture Series 2014 • Turn to academia for the answer • Is not always shared  • But is somewhere between 5 and 9 points How long should a scale be?
  • 96. Pete Cape, SSI, UK April Lecture Series 2014 • Turn to academia for the answer • Is not always shared  • But is somewhere between 5 and 9 points • Jon Krosnick says: • Unipolar: 5 points • Bipolar: 7 points How long should a scale be?
  • 97. Pete Cape, SSI, UK April Lecture Series 2014 • Turn to academia for the answer • Is not always shared  • But is somewhere between 5 and 9 points • Jon Krosnick says: • Unipolar: 5 points • Bipolar: 7 points • Why? • Maximises Reliability and Validity How long should a scale be?
  • 98. Pete Cape, SSI, UK April Lecture Series 2014 What is reliability and validity? • Reliability — Measures the same thing in the same way every time
  • 99. Pete Cape, SSI, UK April Lecture Series 2014 What is reliability and validity? • Reliability — Measures the same thing in the same way every time • Validity — Measures what you think it measures
  • 100. Pete Cape, SSI, UK April Lecture Series 2014 What was Unipolar and Bipolar? “Unipolar” One direction from zero to lots he middle and surfeit at the ends
  • 101. Pete Cape, SSI, UK April Lecture Series 2014 What was Unipolar and Bipolar? “Unipolar” One direction from zero to lots “Bipolar” Opposites with indifference in the middle he middle and surfeit at the ends
  • 102. Pete Cape, SSI, UK April Lecture Series 2014 Scales Unipolar: • Extremely x • Very x • Somewhat x • Slightly x • Not at all x (an) academic conclusion
  • 103. Pete Cape, SSI, UK April Lecture Series 2014 Scales Unipolar: • Extremely x • Very x • Somewhat x • Slightly x • Not at all x Bipolar: • Extremely x • Very x • Somewhat x • Neither x nor un-x • Somewhat un-x • Very un-x • Extremely un-x (an) academic conclusion
  • 104. Pete Cape, SSI, UK April Lecture Series 2014 Scales Unipolar: • Extremely x • Very x • Somewhat x • Slightly x • Not at all x Bipolar: • Extremely x • Very x • Somewhat x • Neither x nor un-x • Somewhat un-x • Very un-x • Extremely un-x And this is what I recommend when asked.... btw: you can’t mix bi- and unipolar scales (an) academic conclusion
  • 105. Pete Cape, SSI, UK April Lecture Series 2014 Scales • Design issues • Display issues
  • 106. Pete Cape, SSI, UK April Lecture Series 2014 Where to put “don’t know” • Visual and Conceptual midpoint
  • 107. Pete Cape, SSI, UK April Lecture Series 2014 Where to put “don’t know” • Visual and Conceptual midpoint Conceptual midpoint Visual midpoint
  • 108. Pete Cape, SSI, UK April Lecture Series 2014 Where to put “don’t know” • Visual and Conceptual midpoint
  • 109. Pete Cape, SSI, UK April Lecture Series 2014 Where to put “don’t know” • Visual and Conceptual midpoint Visual midpoint
  • 110. Pete Cape, SSI, UK April Lecture Series 2014 Where to put “don’t know” • Visual and Conceptual midpoint • Skews data towards “disagree” Conceptual midpoint Visual midpoint
  • 111. Pete Cape, SSI, UK April Lecture Series 2014 Where to put “don’t know” • Visual and Conceptual midpoint Conceptual midpoint Visual midpoint
  • 112. Pete Cape, SSI, UK April Lecture Series 2014 What order to present? Disagree Strongly Disagree Slightly Neither Agree nor Disagree Agree Slightly Agree Strongly
  • 113. Pete Cape, SSI, UK April Lecture Series 2014 What order to present? Disagree Strongly Disagree Slightly Neither Agree nor Disagree Agree Slightly Agree Strongly Order bias
  • 114. Pete Cape, SSI, UK April Lecture Series 2014 What order to present? Disagree Strongly Disagree Slightly Neither Agree nor Disagree Agree Slightly Agree Strongly Order bias Acquiescence bias
  • 115. Pete Cape, SSI, UK April Lecture Series 2014 What order to present? Disagree Strongly Disagree Slightly Neither Agree nor Disagree Agree Slightly Agree Strongly Order bias Acquiescence bias Central tendency
  • 116. Pete Cape, SSI, UK April Lecture Series 2014 What order to present? Disagree Strongly Disagree Slightly Neither Agree nor Disagree Agree Slightly Agree Strongly Order bias Acquiescence bias Central tendency Agree Strongly Agree Slightly Neither Agree nor Disagree Disagree Slightly Disagree Strongly Central tendency Order bias Acquiescence bias
  • 117. Pete Cape, SSI, UK April Lecture Series 2014 Where to put the answer box • Next to answers • 22% failure rate on matching gender to panel record!
  • 118. Pete Cape, SSI, UK April Lecture Series 2014 Where to put the answer box • Next to answers • 22% failure rate on matching gender to panel record! • Left right processing • 98% of that: Males coded as Females
  • 119. Pete Cape, SSI, UK April Lecture Series 2014 Scales • Maximises reliability and validity • Minimises bias • Prevents order error • Not just “the way we always do it…”
  • 120. Pete Cape, SSI, UK April Lecture Series 2014 Scales • Maximises reliability and validity • Minimises bias • Prevents order error • Not just “the way we always do it…” • But isn’t it old-fashioned?
  • 121. Pete Cape, SSI, UK April Lecture Series 2014 Sliders with visual elements
  • 122. Pete Cape, SSI, UK April Lecture Series 2014 129125 77 25 4 0 50 100 150 200 250 Different styles of slider scale
  • 123. Pete Cape, SSI, UK April Lecture Series 2014 Different styles of slider scale 114 8892 17 1 0 50 100 150 200 250
  • 124. Pete Cape, SSI, UK April Lecture Series 2014 Different styles of slider scale 97 21 27 8 1 0 50 100 150 200 250
  • 125. Pete Cape, SSI, UK April Lecture Series 2014 4 10 124 4751 0 50 100 150 200 250 Different styles of slider scale
  • 126. Pete Cape, SSI, UK April Lecture Series 2014 Different distributions, same means? UK USA Germany China traditional - raw data 2.27 2.32 2.50 3.79 traditional - rescored 2.60 2.28 2.50 3.87 slider with all labels 2.56 2.64 2.74 3.89 slider with all tick marks 2.33 2.35 2.46 3.61 slider with no tick marks 2.32 2.39 2.29 3.62 slider with no tick marks, score showing 2.34 2.44 2.33 3.56
  • 127. Pete Cape, SSI, UK April Lecture Series 2014 Grids and Matrices • Q: Do we have these in telephone research? • Q: In Face to Face?
  • 128. Pete Cape, SSI, UK April Lecture Series 2014 Grids and Matrices • Q: Do we have these in telephone research? • Q: In Face to Face? • A: No • Q: Where did we have them?
  • 129. Pete Cape, SSI, UK April Lecture Series 2014 Grids and Matrices • Q: Do we have these in telephone research? • Q: In Face to Face? • A: No • Q: Where did we have them? • A: postal research
  • 130. Pete Cape, SSI, UK April Lecture Series 2014 Item-in-a-series
  • 131. Pete Cape, SSI, UK April Lecture Series 2014 Item-in-a-series “When one has to ask a series of questions that use the same answer categories it is convenient to combine them into a [grid/matrix format]. Combining them into this ‘item-in-a- series’, with a common introduction that defines the general question and response format, eliminates considerable redundancy with regard to stating questions, it also saves considerable space”
  • 132. Pete Cape, SSI, UK April Lecture Series 2014 Item-in-a-series “but do it carefully”
  • 133. Pete Cape, SSI, UK April Lecture Series 2014 Item-in-a-series “but do it carefully” • Items are now in a comparative framework • Visual rendering makes them a unit “if the sponsor wants individuals to contemplate each item separately, it is advisable to present each of them as individual items”
  • 134. Pete Cape, SSI, UK April Lecture Series 2014 Item-in-a-series Should these should be the same?
  • 135. Pete Cape, SSI, UK April Lecture Series 2014 Item-in-a-series Shouldn’t these have been the same?
  • 136. Pete Cape, SSI, UK April Lecture Series 2014 | 136 | What else do grids do? • Encourage you to go too fast
  • 137. Pete Cape, SSI, UK April Lecture Series 2014 Speed of grid completion •What sort of processing is being done at this speed? | 137 |
  • 138. Pete Cape, SSI, UK April Lecture Series 2014 | 138 | What else do grids do? • Encourage you to go too fast • Process the underlying latent construct, not the items — Straightlining
  • 139. Pete Cape, SSI, UK April Lecture Series 2014 | 139 | What else do grids do? • Encourage you to go too fast • Process the underlying latent construct, not the items — Straightlining • Miss the subtleties — Heuristic answering
  • 140. Pete Cape, SSI, UK April Lecture Series 2014 Direct instruction at item 39/40 – pass/fail | 140 |
  • 141. Pete Cape, SSI, UK April Lecture Series 2014 Which heuristic was used? | 141 | answer at trap
  • 142. Pete Cape, SSI, UK April Lecture Series 2014 Which heuristic was used? | 142 | answer at trap
  • 143. Pete Cape, SSI, UK April Lecture Series 2014 And those left over? | 143 |
  • 144. Pete Cape, SSI, UK April Lecture Series 2014 And those left over? | 144 |
  • 145. Pete Cape, SSI, UK April Lecture Series 2014 | 145 | Anything else wrong with grids? • Yes, we use the wrong scale for the construct
  • 146. Pete Cape, SSI, UK April Lecture Series 2014 Beware of the agree-disagree question • Make your scale answers the appropriate ones To what extent to you agree or disagree that this training course is extremely useful? Agree extremely Very much agree Somewhat agree Neither agree nor disagree Somewhat disagree Very much disagree Disagree extremely
  • 147. Pete Cape, SSI, UK April Lecture Series 2014 2 stage process 1. Decide how useful you find this
  • 148. Pete Cape, SSI, UK April Lecture Series 2014 2 stage process 1. Decide how useful you find this 2. Translate into agree-disagree, hopefully: Extremely useful = Agree extremely Very useful = Very much agree Somewhat useful = Somewhat agree Neither useful not disuseful = Neither agree nor disagree Somewhat disuseful = Somewhat disagree Very disuseful = Very much disagree Extremely disuseful = Disagree extremely
  • 149. Pete Cape, SSI, UK April Lecture Series 2014 2 stage process 1. Decide how useful you find this 2. Translate into agree-disagree, hopefully: • Disuseful? Extremely useful = Agree extremely Very useful = Very much agree Somewhat useful = Somewhat agree Neither useful not disuseful = Neither agree nor disagree Somewhat disuseful = Somewhat disagree Very disuseful = Very much disagree Extremely disuseful = Disagree extremely
  • 150. Pete Cape, SSI, UK April Lecture Series 2014 2 stage process • Reality (maybe): Extremely useful = Agree extremely Very useful = Very much agree Somewhat useful = Somewhat agree Slightly useful = Somewhat agree Not at all useful = Neither/nor, Somewhat disagree,Very much disagree,Disagree extremely
  • 151. Pete Cape, SSI, UK April Lecture Series 2014 How to measure attitudes/constructs • Work out the underlying construct — i.e. what are you measuring
  • 152. Pete Cape, SSI, UK April Lecture Series 2014 How to measure attitudes/constructs • Work out the underlying construct — i.e. what are you measuring • Decide how it is measured — What words — Unipolar or Bipolar
  • 153. Pete Cape, SSI, UK April Lecture Series 2014 How to measure attitudes/constructs • Work out the underlying construct — i.e. what are you measuring • Decide how it is measured — What words — Unipolar or Bipolar • Check your measurement gaps are equal • Ask question..
  • 154. Pete Cape, SSI, UK April Lecture Series 2014 How to measure attitudes/constructs • Work out the underlying construct — i.e. what are you measuring • Decide how it is measured — What words — Unipolar or Bipolar • Check your measurement gaps are equal • Ask question.. • More work for you, less work for respondent • And more valid and reliable
  • 155. Pete Cape, SSI, UK April Lecture Series 2014 Construct Specific Scale How useful, if at all, did you find this training course? Extremely useful Very useful Somewhat useful Slightly useful Not at all useful • As you read the question your answer is forming….. ?
  • 156. Pete Cape, SSI, UK April Lecture Series 2014 Construct Specific Scale How useful, if at all, did you find this training course? Extremely useful Very useful Somewhat useful Slightly useful Not at all useful • As you read the question your answer is forming….. • Order effects?
  • 157. Pete Cape, SSI, UK April Lecture Series 2014 Construct Specific Scale Extremely useful Very useful Somewhat useful Slightly useful Not at all useful
  • 158. Pete Cape, SSI, UK April Lecture Series 2014 Construct Specific Scale Extremely useful Very useful Somewhat useful Slightly useful Not at all useful Expectation: up = good (Tourangeau)
  • 159. Pete Cape, SSI, UK April Lecture Series 2014 Construct Specific Scale Extremely useful Very useful Somewhat useful Slightly useful Not at all useful Expectation: up = good (Tourangeau) Answer speed = faster (Christian et al) No order bias found
  • 160. Pete Cape, SSI, UK April Lecture Series 2014 Manipulating people by accident • Because we want to make it more fun
  • 161. Pete Cape, SSI, UK April Lecture Series 2014 Please select your most preferred holiday type… Summer Beach Winter City Break Adventure
  • 162. Pete Cape, SSI, UK April Lecture Series 2014 Please select your most preferred holiday type… Summer Beach Winter City Break Adventure
  • 163. Pete Cape, SSI, UK April Lecture Series 2014 Please select your most preferred holiday type… Summer Beach Winter City Break Adventure
  • 164. Pete Cape, SSI, UK April Lecture Series 2014 The results 58% 60% 36% 0% 20% 40% 60% 80% 100% “Summer Beach”
  • 165. Pete Cape, SSI, UK April Lecture Series 2014 Motivation
  • 166. Pete Cape, SSI, UK April Lecture Series 2014 Motivation Self-Determination Theory (Deci & Ryan)
  • 167. Pete Cape, SSI, UK April Lecture Series 2014 Motivation • people are active ‘organisms’ • having tendencies toward psychological growth and development • who strive to master ongoing challenges • and to integrate their experiences into a coherent sense of self. • the social context can either support or thwart the natural tendencies Self-Determination Theory (Deci & Ryan)
  • 168. Pete Cape, SSI, UK April Lecture Series 2014 Motivation under Self-Determination Theory A continuum that relates to the task in the social context, not a person
  • 169. Pete Cape, SSI, UK April Lecture Series 2014 Motivation under Self-Determination Theory A continuum that relates to the task in the social context, not a person Regulation Introjection Identification Integration to obtain an externally imposed reward pressure to avoid guilt or to enhance self-esteem conscious valuing of activity assimilated to self task is not done for it’s own reward
  • 170. Pete Cape, SSI, UK April Lecture Series 2014 Motivation under Self-Determination Theory A continuum that relates to the task in the social context, not a person Regulation Introjection Identification Integration to obtain an externally imposed reward pressure to avoid guilt or to enhance self-esteem conscious valuing of activity assimilated to self task is not done for it’s own reward Where do most market research respondents fall?
  • 171. Pete Cape, SSI, UK April Lecture Series 2014 Motivation under Self-Determination Theory A continuum that relates to the task in the social context, not a person Regulation Introjection Identification Integration to obtain an externally imposed reward pressure to avoid guilt or to enhance self-esteem conscious valuing of activity assimilated to self task is not done for it’s own reward Where do most market research respondents fall?
  • 172. Pete Cape, SSI, UK April Lecture Series 2014 Motivation under Self-Determination Theory Task Outcomes Regulation Introjection Identification Integration little interest, value or effort more effort, anxiety, poor coping with failure enjoyment, good coping skills greater enjoyment, competency task is not done properly interest, enjoyment highly competent Where would we like respondents to be?
  • 173. Pete Cape, SSI, UK April Lecture Series 2014 Motivation under Self-Determination Theory Task Outcomes Regulation Introjection Identification Integration little interest, value or effort more effort, anxiety, poor coping with failure enjoyment, good coping skills greater enjoyment, competency task is not done properly interest, enjoyment highly competent Where would we like respondents to be?
  • 174. Pete Cape, SSI, UK April Lecture Series 2014 Moving people along the motivation continuum • Foster feelings of: — Autonomy – you are free to do this or not as you choose — Competence – you are good at this — Relatedness – people like you do this — Value – what you are doing has meaning
  • 175. Pete Cape, SSI, UK April Lecture Series 2014 Moving people along the motivation continuum • Foster feelings of: — Autonomy – you are free to do this or not as you choose — Competence – you are good at this — Relatedness – people like you do this — Value – what you are doing has meaning • Perhaps this was what the interviewer used to do? • This is what Gamification helps do
  • 176. Pete Cape, SSI, UK April Lecture Series 2014 Motivating respondents Thank you very much for choosing to do this important survey. As one of our most expert survey takers we have selected you to help on this survey. People like you all over the country are taking part in this survey. Your answers and those of everyone else will help improve the services our client provides to customers like you. Please press the > button when you are ready to continue with your survey.
  • 177. Pete Cape, SSI, UK April Lecture Series 2014 Motivating respondents Thank you very much for choosing to do this important survey. As one of our most expert survey takers we have selected you to help on this survey. People like you all over the country are taking part in this survey. Your answers and those of everyone else will help improve the services our client provides to customers like you. Please press the > button when you are ready to continue with your survey. • No mention of rewards • No threats
  • 178. Pete Cape, SSI, UK April Lecture Series 2014 Summary • The questionnaire is a mixture of art and science • The art of writing well, of motivating respondents • The science of writing precisely, avoiding bias • Everything has meaning • You are supposed to know what it means • There are better ways of asking questions • There are better ways of collecting answers • You might as well do the best you can
  • 179. Pete Cape, SSI, UK April Lecture Series 2014 Summary • Questionnaire writing • (and understanding why it is important) • Sets us apart from others • And has real value
  • 180. Pete Cape, SSI, UK April Lecture Series 2014 Summary - Principles • Respondents know they are in an experiment • They think everything they see, hear or read is deliberate, and has meaning • They want to please • Questionnaire motivates the respondent to provide — complete answers — accurate answers • It communicates to the respondent • It helps the respondent work out their answers • It makes the respondents task easy
  • 181. Pete Cape, SSI, UK April Lecture Series 2014 And guess what? • If it was easy do you think there would be textbooks on it?
  • 182. Pete Cape, SSI, UK April Lecture Series 2014 Thank you! Pete Cape Global Knowledge Director, SSI 2014 Platinum Sponsor April Series Sponsor Lecture Sponsor
  • 183. Pete Cape, SSI, UK April Lecture Series 2014 The Sponsors for this Event If you are interested in sponsoring a future NewMR event Email Michele.Poynter@TheFuturePlace.com 2014 Platinum Sponsor April Series Sponsor Lecture Sponsor
  • 184. Pete Cape, SSI, UK April Lecture Series 2014 Q & A Ray Poynter The Future Place Pete Cape SSI 2014 Platinum Sponsor April Series Sponsor Lecture Sponsor
  • 185. Pete Cape, SSI, UK April Lecture Series 2014 | 186 | surveysampling.com info@surveysampling.com pete.cape@surveysampling.com