Mixing the Right Sample Ingredients
•
1 j'aime•264 vues
At the Advertising Research Foundation’s (ARF) 2011 annual re:think convention, a key issues forum presentation was held entitled Mixing the Right Sample Ingredients. The presentation was given by Jackie Lorch, VP of Global Knowledge Management for Survey Sampling International. The presentation discussed which factors to blend and emphasized the importance of implementing multi-source testing.
Recommandé
Contenu connexe
Similaire à Mixing the Right Sample Ingredients
Similaire à Mixing the Right Sample Ingredients (20)
Plus de The Advertising Research Foundation
Plus de The Advertising Research Foundation (20)
Dernier
Dernier (20)
Mixing the Right Sample Ingredients
- 2. Mixing the Right Sample Ingredients A New Source Recipe Jackie Lorch VP, Global Knowledge Management Survey Sampling International
- 3. What is the Problem We Are Trying to Solve? 1. A systematic problem with online access panels
- 4. What is the Problem We Are Trying to Solve? 2. A scarce resource
- 5. Many Reasons People Are Not in Panels Why have you not joined an Why do you no longer belong to an online research panel?1 online research panel?2 I don’t want any more emails 7% Too many surveys 4% I think they are scams 6% Not enough surveys 27% It’s a waste of time 3% I never qualified for a survey 44% Not enough time to take surveys 11% Poor respondent experience 4% Never heard of an online research panel 20% Not enough time to take surveys 17% Don’t want to commit to a panel 22% Didn’t want to commit to a panel anymore 8% Never had any opportunities to join a panel 37% Other 8% Other 6% 1. Asked of people online who have never joined a research panel 2. Asked of people online, on a panel previously, not currently Multiple answers allowed
- 6. What is the Problem We Are Trying to Solve?
- 7. What is the Problem We Are Trying to Solve?
- 8. A Better Breed of Online Sample
- 9. Real Differences in Behavior Own an MP3 player
- 10. Why Not Use Socio-Demographics? • Demographics may not be the most helpful or relevant stabilizers • Demographic quotas work if the stratification chosen is relevant to questionnaire topic • A product may be liked across all demographics
- 11. Which are the right quotas? Sample 1 Sample 2 Balanced on: Balanced on: Age Ethnicity Gender Region
- 12. From Sources to People • Using thousands of sources • Metric must be at “person” level
- 13. Beyond Socio-Demographics With effective stabilization variables, no need to maintain same blend of sources project-to-project
- 14. Factors for the Blend Broad set of factors defining groups of people, spanning multiple disciplines Personality traits Need for cognition (Cacioppo et al) Music preferences (Rentfrow) Neurographics measures Cognitive ability Propensity factors e.g., to participate, (Kahneman and Frederick) share, risk averse, attitude to privacy Geographic / personality alignment Chronotypes, “lark” or “owl” (Gosling and Rentfrow) (Roenneberg) Social Values. Schwarz has international Disruption/orienting reflex measure, benchmarks habituation disruption
- 15. Factors for the Blend • Questions created around 162 variables • Tested factor performance against questions on technology, hobbies, interests, brand preference, loyalty, ad awareness • Variables tested iteratively on power to “move the needle” on dependent variables • 14 variables identified as explaining more variance than socio-demographics alone; from these created 8 clusters to control
- 16. Multi-Source Testing • Broad set of factors defining groups of people, spanning multiple disciplines • Factors included such topics as attitudes to trust, gender roles, online behavior, personality traits, ideas and beliefs • Results varied by source
- 17. Results Closer to “Truth” On measures where benchmarks are available, blend comes closer to it Social Media iPod/MP3 Smartphone (Facebook or Ownership Ownership MySpace) SSI SurveySpot panel 37 16 60 Benchmark 43 c.20-22 69 SSI Blend 43 20 68 Benchmark sources: Pew Research; comScore; Harris Interactive
- 20. Lock and Control the Stream
- 21. Simulation Without Control MP3 ownership – uncontrolled
- 22. Simulation Holding Clusters Constant MP3 ownership – controlled
- 23. Practical Steps When Blending • Know your sources • Use a consistent blend for the entire project • Use benchmarks and calibrate • Understand blending techniques being used
- 24. In Summary • Multi-sourcing is the future, offering superior sample • We can keep multi-sourced samples consistent by pre-profiling participants • Standard socio-demographics aren’t enough • A concise list of variables can be used for a broad range of research subjects • Participant experience is key