Chapter 4: Alternatives to Experimentation:
Surveys and Interviews
Survey research: useful way of obtaining information about people’s opinions, attitudes, preferences, and behaviors simply by
asking
Examples: telephone surveys, election polls, television ratings, and customer satisfaction surveys
-gather data about experiences, feelings, thoughts, and motives that are hard to observe directly
-useful collecting data about sensitive subjects because they can be given anonymously so people will answer more
honestly
-useful for making inferences about behavior but they do not allow for testing hypotheses about causal relationships
directly
-used in conjunction with other research designs
-gather large amounts of data efficiently
-low in manipulation; range from low to high imposition units
-responses can be limited (yes or no questions) or free response
Two most common types of surveys:
-Written questionnaires: handed out or sent through mail

-Interviews: face-to-face or on the phone; in person interviews can be individual or group
*generalizability of surveys depends on how subjects were selected
Constructing Surveys
-Step 1: map out your research objectives, making them as specific as possible; (ex: objective is to measure the attitudes
of psych students toward animal research in psychology; ask specific questions about things like animal rights, animal welfare,
benefits to humanity, etc); to get ideas for objectives, look at previous research
-Step 2: design the survey items; decide how you are going to address the imposition of units (do you want long, free
responses or a limited number of alternatives)
-closed questions (structured questions): ex: do you want to smoke? or rate on a scale of 1-10; must be answered by
one of a limited number of alternatives; closed questions are easier to quantify (easier to give a percent or number
of how many children answered each of the four possible questions about cartoons)
-open-ended questions (open questions): solicit information about opinions and feelings by asking the question in
such a way that the person must respond with more than a yes, no, or 1-10 rating (ex: what are your feelings about
airport security); can be used to clarify or expand answers to closed questions (combination questions --> ex: how
much time do you spend watching cartoons, less than an hour, 1-2 hours, 2-4 hours, 5+ hours followed by
why do you watch? What do you think about the characters who hit each other? Etc)
-content analysis: the process of quantifying open question answers; similar to coding behaviors using systematic
observation techniques; ex: for the question about characters hitting each other, a content analysis may be what
kinds of things may cause you to hit someone? Divide those responses in categories (someone said
something to me, someone looked at me funny, etc

Double-barreled questions (compound questions): questions that ask for responses about two (or more) different ideas in the
same questions; should be avoided; ex: Do you like strawberries and ice cream?  You like strawberries but not ice
cream so you could not answer this question
Exhaustive: response choices need to contain all possible options; ex: what exercise do you do the most? Play a sport, walk, or
jog  your favorite is yoga and that is not listed
-you can use “other” as an option but do so only if it would be chosen rarely because then it is harder to interpret results
(it would be difficult to interpret answers with the option of "other" because there would be too many different responses); if a
question requires 6 or more response options, using an open-ended question would be better
Level of measurement: the kind of scale used to measure a response for a closed question; different statistical tests are required
for different levels of measurement; four kinds of scales:
-nominal: simplest level of measurement; classifies response items into two or more distinct categories (that can be
named) on the basis of a common feature; cannot be quantified; ex: true-false test, answer is only one of those; lowest level of
measurement because it provides not information about magnitude (ex: political affiliation you belong to one party but
no party is better than the other)
-ordinal scale: rank ordering of response item; magnitude of each value is measured in the form of ranks; ex: ranking
presidential candidates; gives a relative order of preference but is not precise (with presidential polls, it tells who is most/least
popular but not exactly how popular they are)
-interval scale: measures the magnitude or quantitative size using measures with equal intervals between the vales; no
true zero point (the true absence of any measurable temperature); ex: temperature in Fahrenheit; 40 degrees is not twice as hot
as 20 degrees because the intervals between values are equal
-ratio: highest level of measurement; equal intervals between all values and a true zero point; measurements of physical
characteristics like height and weight can be measured with ratio scales

*The best type of scale to use will depend on two things: the nature of the variable you are studying and how much
measurement precision you desire* (presidential candidate example: you may only want to know the candidates marital status
(nominal) or how many years candidate has been married (ratio
*SCALING TECHNIQUES*
-semantic differential: evaluating variable on a number of dimensions; two adjectives (ex: positive and negative)
separated by a scale (usually consisting of 7 blanks)
-Likert: present a positively worded statement with a negatively worded statement (strongly agree or strongly
disagree)
Continuous dimension: when variables lend themselves to different levels of measurement; traits, attitudes, and preferences are
all continuous; ex: trait of sociability can range from very unsociable to very sociable (each person falls somewhere on that
dimension)
*when selecting a level of measurement that all "fit" equally well, choose the highest level possible because it provides more
information about the response)*
Important considerations for Survey items:
-get subjects involved right away by asking interesting questions
-the first question should be something that people will not mind answering; should have these characteristics:
-relevant to the central topic
-easy to answer
-interesting
-answerable by most respondents

-closed format
-the first few questions should be ones that subjects do not have to think about (no open ended), are able to answer
without saying “I don’t know”, and will think are relevant to the topic of the survey
*make sure questions are not value laden --> do not word your questions in ways that would make a positive (or negative)
response seem embarrassing or undesirable; ex: do you believe doctors should be able to kill unborn babies in the first trimester
or do you believe doctor should be able to terminate a pregnancy in the first trimester --> first question is difficult to say yes to
due to the negative wording
Response styles: tendencies to respond to questions or test items in specific ways, regardless of the content; ex: people differ in
response styles, such as willingness to answer, position preferences, and yea-saying or nay-saying
-willingness to answer: comes into play whenever questions require specific knowledge about facts or issues; when
unsure, people leave questions blank or guess which makes results harder to interpret
-position preference: occurs with multiple choice questions; ex: when in doubt you always choose b; to avoid, vary the
arrangement of correct responses (ex: in a survey with questions about attitudes towards abortion, do not always put "pro-
choice" as option B
-manifest content: the plain meaning of the words that actually appear on the page; ex: have you ever visited another
country literally means have you ever visited another country
-yea-sayers: apt to agree with a question regardless of its manifest content
-nay-sayers: tend to disagree no matter what they are asked
^^can be avoided by designing the questions that force the subject to think more about the answer; ex: do you agree
or disagree that the cost of living has gone up in the last year? or In your opinion, have prices gone up, gone down,
or stayed about the same the past year, or don't you know? --> building specific content into the options like in the
second question makes people think harder about their choice

-to avoid yea and nay-sayers, you can use the unfounded optimism inventory (underline the optimistic answer
which can be yes or no and it forces yea/nay-sayers to choose either of the options; ex: i know that everything will
be alright: YES NO; I always stand in the slowest line at the bank YES NO
-once the questions have been designed they need to be pretested
-context effects: (caught through pretesting); sometimes the position of a questions or where it falls within the question
order can influence how the question is interpreted; likely when two questions are related
Buffer items: used to separate questions that are similar; questions that are unrelated to both of the related questions
latent content: the way people interpret what you are trying to ask; subjects may not fully understand
Collecting Survey Data:
Questionnaires: (if written) instructions should be simple and clear; if possible, let subjects fill out in private or
anonymously if possible
Mail surveys: include a cover letter, make sure questionnaire and return procedure protects anonymity, include return
envelope and stamp; holding a drawing prize or compensation for return can increase return rates; keep track of who does not
return questionnaires; send a second survey to people who did not respond (it can increase response rate)
Telephone surveys: most widely used method, may not get completely forthright answers; usually not open ended
Internet surveys
Interviews: one of the best ways to get high-quality survey data; expensive; take twice as long to conduct
-structured interview: the same questions are asked in the same way each time; provide more usable, quantifiable data

-unstructured interview: more free flowing; interviewer is free to explore issues as they come up; info may not be
usable for statistics
Focus groups: face to face technique used less often for data collection; good for pretesting; groups have similar
characteristics (all women, all black, etc); group is brought together by an interviewer called a "facilitator"; facilitator wants
group to answer a set of open-ended questions but the discussion is not limited
-response rate and representativeness is effected with each different one
*Evaluating surveys and survey data
Reliability: the extent to which the survey is consistent and repeatable; survey is reliable if responses to similar questions in the
survey should be consistent, the survey should generate very similar responses if it is given to survey-givers, and the survey
should generate very similar responses if it is given to the same person more than once
Validity: the extent to which a survey actually measures the intended topic; does the survey measure what you want it to
measure? does performance on the survey predict actual behavior? does it give the same results as other surveys designed to
measure similar topics? do the individual survey items fairly capture all the important aspects of the topic?; pretesting questions
increases validity
Sampling: deciding who or what the subjects will be and, then, selecting them
Population: all people, animals, or objects that have at least one characteristic in common; ex: all undergraduate students
Sample of subjects: a group that is a subset of the population of interest
Representativeness: how closely the sample mirrors the large population
Probability sampling: selecting subjects in such a way that the odds of their being in the study are known or can be calculated;
begin by defining the sample you want to study (ex: women born in 1975 now living in Seattle), then choose an unbiased
method for selecting the subjects (random selection: any member of the population has an equal opportunity to be selected)

-Simple random sampling: most basic form of probability sampling; a portion of the whole population is selected in an
unbiased way; all members of the population being studied must have an equal chance of being selected
-Systematic random sampling: all members of the population are known and can be listed in an unbiased way; a research
picks the nth person; n is determined by size of population and the desired sample size
-Stratified random sampling: used when populations have distinct subgroups; obtained by randomly sampling from people in
each subgroup in the same proportions as they exist in the population
-example: majors at Clemson; 50% of the students are engineering majors --> our sample of the population has to have
50% engineers to represent the general population
-Cluster sampling: sample entire clusters or naturally occurring groups that exist within the population; used if individual
sampling is impossible due to cost or too large of a population; less reliable (example of clusters: zip code areas, school
districts, etc)
-example: if you want to sample students from the business of behavioral science, you just give everyone who takes intro
to psych a survey
Nonprobability sampling: subjects are not chosen at random
Quota sampling: select samples through predetermined quotas that reflect the makeup of the population
Convenience sampling: using any groups who happen to be available
Purposive sampling: when nonrandom samples are selected because the individuals reflect a specific purpose of the study
-ex: comparing new training program for employees in two departments --> select the employees of those two departments
Snowball sampling: researcher locates one or a few people who fit the criteria and asks these people to find more people