CARMA Summer Workshop

1. Sampling for Internet Research Carma Research Module Jeff Stanton

2. May 15-17, 2008 Internet Data Collection Methods (Day 2-2) Universe, Population, Frame Universe: U.S. Residents, 18+ Population: Males, 18+

3. May 15-17, 2008 Internet Data Collection Methods (Day 2-3) Universe, Population, Frame, Sample Universe: U.S. Residents, 18+ Population: Males, 18+ Sample Malhotra, N. K. (1999). Marketing research: An applied orientation. (International 3rd edition). London: Prentice Hall.

4. May 15-17, 2008 Internet Data Collection Methods (Day 2-4) Multiple Sources of Error(http://www.idready.org/courses/2005/spring/survey_SamplingFrames.pdf)

5. May 15-17, 2008 Internet Data Collection Methods (Day 2-5) Sampling for Internet-Based Studies How to find, buy, or build this list? OpenRecruitment Final Sample Email Invitation

6. May 15-17, 2008 Internet Data Collection Methods (Day 2-6) Probability vs. Non-Probability Superior Statistical“Projectability” RDD – May provide truly random sampleIntra-Org Census as Sampling Frame Limited or no Statistical “Projectability” Assumes triangulationis required Generalizability basedin part on theory foundation

7. May 15-17, 2008 Internet Data Collection Methods (Day 2-7) Two types of Non-Probability Sampling Non-purposive Convenience sampling Open recruitment Student samples in place of actual population Snowball or chain (can also be purposive) Purposive Expert, Quota, Politically important Heterogeneity, Extreme/deviant, maximum variation, intensity Homogeneity, modal instance, critical case, criterion, Theory-based, disconfirming Patton, M. Q. (1990). Qualitative evaluation and research methods (2nd ed.). Newbury Park, CA: Sage Publications.

8. May 15-17, 2008 Internet Data Collection Methods (Day 2-8) Internet Sources of Samples Non-purposive Email harvesting Snowball, chain emails, or IM-Buddy lists Banner ads Site-based polls Site-based links Class rosters Social networking sites Purposive Organizational membership lists Listservs of interest groups or professional organizations Purchased sample frames Standing panels Amazon MechTurk

9. Amazon Mechanical Turk Some pundits have commented that you can pay $0.03 per survey question. Responses come quickly: more than one response per minute in some cases.

10. Mturk Has Built-in Survey Capability

11. Mturk Provides Quick Responses

12. Internet Data Collection Methods (Day 2-12) Criticisms of Internet Samples Digital Divide: respondents from different communities have differing access to necessary tools Non-sampled respondents providing responses Sampled respondents providing bogus responses due to lack of proctoring Amorphous sampling frame (e.g., due to posting an open link) Internet users per 100 people

13. May 15-17, 2008 Internet Data Collection Methods (Day 2-13) A response to these problems:Mixed Mode Design Telephone response rates have dropped from the 70-80% in the mid-20th century to less than 40%; mail response rates have always been low; Internet response rates have declined Solution: Combine mail, phone, Internet, and other modes Use best available sampling for each mode Instrument design presents another challenge: Uni-mode or mode specific

14. May 15-17, 2008 Internet Data Collection Methods (Day 2-14) Preferred Strategy: Mixed Mode Sampling KnownPopulation Values Start with Theory Sampling Frame 1: Mail or Other Sample Answer: Who,When, Why Triangulate:Calibrate & Compare Sampling Frame 2: Internet Sample Develop MultipleSample Sources andResearch Modes