This presentation can be a reference for those taking up research particularly psychology students.

1. Alternatives to
Experimentation:
Surveys and InterviewsBy Maria Jezza C. Ledesma

2. Chapter
Objectives
•Learn about more nonexperimental
techniques for studying behavior: survey
and interview research
•Learn the factors involved in designing
questionnaires and devising good questions
•Using standardized tests
•Learn how to administer questionnaires and
conduct interviews
•Learn how focus groups work
•Learn the pros and cons of different
sampling techniques
2

3. Survey Research
⊙ Useful way of obtaining
information about people’s
opinions, attitudes,
preferences, and behaviors
simply by asking
3
⊙Examples: telephone
surveys, election polls,
television ratings, and
customer satisfaction
surveys

4. 4

5. 5

6. 6

7. 7

8. ⊙Gather data about experiences,
feelings, thoughts, and motives that are
hard to observe directly
⊙Useful collecting data about sensitive
subjects because they can be given
anonymously so people will answer
more honestly
⊙Useful for making inferences about
behavior but they do not allow for
testing hypotheses about causal
relationships directly
8
SURVEYS

9. ⊙Used in conjunction with other
research designs
⊙Gather large amounts of data
efficiently
⊙Low in manipulation; range from
low to high imposition units
⊙Responses can be limited (yes or
no questions) or free response
9

10. Two most
common
types of
surveys
⊙ Written Questionnaires
○ Handed out or sent through
mail
⊙ Interviews
○ Face-to-face or on the phone;
in person interviews can be
individual or group
10

11. 11
• Generalizability of
surveys depends on how
subjects were selected.
👪

12. Constructing
Surveys
12

13. Step
1
⊙ Map out your research objectives,
making them as specific as possible;
○ Ex: objective is to measure the
attitudes of psych students toward
animal research in psychology; ask
specific questions about things like
animal rights, animal welfare,
benefits to humanity, etc); to get
ideas for objectives, look at
previous research
13

14. Step
2
⊙ Design the survey items; decide how you
are going to address the imposition of
units (do you want long, free responses or
a limited number of alternatives)
⊙Closed questions (structured questions)
○ Ex: Do you want to smoke? or rate on a
scale of 1-10; must be answered by one
of a limited number of alternatives;
closed questions are easier to quantify
(easier to give a percent or number of
how many children answered each of
the four possible questions about
cartoons)
14

15. Step
2
⊙ Open-ended questions (open questions)
○ Solicit information about opinions and
feelings by asking the question in such a
way that the person must respond with
more than a yes, no, or 1-10 rating
■ Ex: what are your feelings about airport
security);
o Can be used to clarify or expand answers to
closed questions (combination questions)
 Ex: how much time do you spend
watching cartoons, less than an hour, 1-
2 hours, 2-4 hours, 5+ hours
 followed by why do you watch? What do
you think about the characters who hit
each other? Etc)
15

16. Step
2
⊙ Content Analysis
○ The process of quantifying open
question answers; similar to coding
behaviors using systematic
observation techniques;
○ Ex: for the question about characters
hitting each other, a content analysis
may be what kinds of things may cause
you to hit someone?
■ Divide those responses in
categories (someone said
something to me, someone looked
at me funny, etc)
16

17. Tips for
constructing
questions
⊙Keep it simple
and keep people
involved
17

18. Double-
barreled
questions
(compound
questions)
⊙Questions that ask for responses
about two (or more) different
ideas in the same questions;
should be avoided;
⊙Ex: Do you like strawberries and
ice cream?
○ You like strawberries but not
ice cream so you could not
answer this question
18

19. Exhaustive
⊙Response choices need to
contain all possible
options;
○ Ex: what exercise do you do
the most? Play a sport, walk,
or jog but your favorite is
yoga and that is not listed
19

20. ⊙You can use “other” as an option but
do so only if it would be chosen rarely
because then it is harder to interpret
results (it would be difficult to
interpret answers with the option of
"other" because there would be too
many different responses); if a
question requires 6 or more response
options, using an open-ended
question would be better
20

21. Level of
Measurement
⊙The kind of scale used to
measure a response for a
closed question; different
statistical tests are
required for different
levels of measurement;
21

22. Four
Kinds of
Scales
⊙ Nominal Scale
○ Simplest level of measurement;
classifies response items into two or
more distinct categories (that can be
named) on the basis of a common
feature; cannot be quantified; ex: true-
false test, answer is only one of those;
lowest level of measurement because it
provides not information about
magnitude
○ Ex: political affiliation
■ You belong to one party but no
party is better than the other
22

23. ⊙Ordinal Scale
○ Rank ordering of response item;
magnitude of each value is
measured in the form of ranks;
■ Ex: ranking presidential
candidates; gives a relative
order of preference but is not
precise (with presidential polls,
it tells who is most/least popular
but not exactly how popular they
are)
23

24. ⊙Interval Scale
○ Measures the magnitude or
quantitative size using measures
with equal intervals between the
vales; no true zero point (the true
absence of any measurable
temperature);
○ Ex: temperature in Fahrenheit; 40
degrees is not twice as hot as 20
degrees because the intervals
between values are equal
24

25. ⊙Ratio
○ Highest level of
measurement; equal intervals
between all values and a true
zero point; measurements of
physical characteristics like
height and weight can be
measured with ratio scales
25

26. ⊙The best type of scale to use will
depend on two things:
○ The nature of the variable you are
studying and how much
measurement precision you desire
(presidential candidate example:
you may only want to know the
candidates marital status (nominal)
or how many years candidate has
been married (ratio)
26

27. 27
SCALING
TECHNIQUES

28. ⊙Semantic Differential
○ Evaluating variable on a
number of dimensions;
two adjectives (ex:
positive and negative)
separated by a scale
(usually consisting of 7
blanks)
28

29. ⊙Likert
○ Present a positively worded statement
with a negatively worded statement
(strongly agree or strongly disagree)
⊙Continuous Dimension
○ When variables lend themselves to
different levels of measurement; traits,
attitudes, and preferences are all
continuous;
■ Ex: trait of sociability can range
from very unsociable to very
sociable (each person falls
somewhere on that dimension)
29

30. ⊙When selecting a level of
measurement that all "fit"
equally well, choose the
highest level possible
because it provides more
information about the
response
30

31. Important
Considerations
for Survey
Items
⊙Get subjects involved right away by asking
interesting questions
⊙ The first question should be something that
people will not mind answering; should have
these characteristics:
○ Relevant to the central topic
○ Easy to answer
○ Interesting
○ Answerable by most respondents
○ Closed format
⊙ The first few questions should be ones that
subjects do not have to think about (no open
ended), are able to answer without saying “I
don’t know”, and will think are relevant to the
topic of the survey 31

32. ⊙Make sure questions are not value laden
○ Do not word your questions in ways
that would make a positive (or negative)
response seem embarrassing or
undesirable;
■ Ex: do you believe doctors should
be able to kill unborn babies in the
first trimester or do you believe
doctor should be able to terminate a
pregnancy in the first trimester
● First question is difficult to say
yes to due to the negative
wording
32

33. ⊙Response Styles
○ Tendencies to respond to questions or test
items in specific ways, regardless of the
content;
■ Ex: people differ in response styles,
such as willingness to answer, position
preferences, and yea-saying or nay-
saying
○ Willingness to answer
■ Comes into play whenever questions
require specific knowledge about facts
or issues; when unsure, people leave
questions blank or guess which makes
results harder to interpret
33

34. ○ Position Preference
■ Occurs with multiple choice questions;
ex: when in doubt you always choose b;
to avoid, vary the arrangement of
correct responses
● Ex: in a survey with questions about
attitudes towards abortion, do not
always put "pro-choice" as option B
○ Manifest Content
■ The plain meaning of the words that
actually appear on the page;
● Ex: have you ever visited another
country literally means have you
ever visited another country
34

35. ⊙Yea-sayers
○ Apt to agree with a question
regardless of its manifest
content
⊙Nay-sayers
○ Tend to disagree no matter
what they are asked
35

36. ⊙Can be avoided by designing the
questions that force the subject to think
more about the answer;
○ Ex: do you agree or disagree that the
cost of living has gone up in the last
year? or In your opinion, have prices
gone up, gone down, or stayed
about the same the past year, or
don't you know?
■ Building specific content into the
options like in the second
question makes people think
harder about their choice 36

37. ⊙To avoid yea and nay-sayers, you
can use the unfounded optimism
inventory (underline the optimistic
answer which can be yes or no
and it forces yea/nay-sayers to
choose either of the options;
○ Ex: I know that everything will
be alright: YES NO; I always
stand in the slowest line at the
bank YES NO
37

38. ⊙Once the questions
have been designed
they need to be
pretested
38

39. ⊙ Context Effects
○ (caught through pretesting)
sometimes the position of a
questions or where it falls
within the question order
can influence how the
question is interpreted;
likely when two questions
are related
39

40. ⊙Buffer Items
○ Used to separate
questions that are
similar; questions that
are unrelated to both
of the related
questions
40

41. ⊙Latent Content
○ The way people
interpret what you are
trying to ask; subjects
may not fully
understand
41

42. 42
Collecting
Survey
Data

43. ⊙Questionnaires
○ (if written) instructions should be simple
and clear; if possible, let subjects fill out
in private or anonymously if possible
⊙Mail Surveys
○ Include a cover letter, make sure
questionnaire and return procedure
protects anonymity, include return
envelope and stamp; holding a drawing
prize or compensation for return can
increase return rates; keep track of who
does not return questionnaires; send a
second survey to people who did not
respond (it can increase response rate)
43

44. ⊙Internet Surveys
⊙Interviews
○ One of the best ways to get high-quality
survey data; expensive; take twice as
long to conduct
⊙Structured Interview
○ The same questions are asked in the
same way each time; provide more
usable, quantifiable data
⊙Unstructured Interview
○ More free flowing; interviewer is free to
explore issues as they come up; info
may not be usable for statistics
44

45. ⊙Focus Groups
○ Face to face technique used less
often for data collection; good for
pretesting; groups have similar
characteristics (all women, all black,
etc); group is brought together by an
interviewer called a "facilitator";
facilitator wants group to answer a
set of open-ended questions but the
discussion is not limited
○ Response rate and
representativeness is effected with
each different one 45

46. Evaluating
Surveys and
Survey Data
46

47. Reliability
⊙The extent to which the survey is
consistent and repeatable;
○ Survey is reliable if responses to
similar questions in the survey
should be consistent, the survey
should generate very similar
responses if it is given to survey-
givers, and the survey should
generate very similar responses if it
is given to the same person more
than once
47

48. Validity
⊙The extent to which a survey actually
measures the intended topic;
○ Does the survey measure what you
want it to measure? Does
performance on the survey predict
actual behavior? Does it give the
same results as other surveys
designed to measure similar topics?
Do the individual survey items fairly
capture all the important aspects of
the topic?
○ Pretesting questions increases
validity 48

49. ⊙Sampling
○ Deciding who or what the subjects will
be and, then, selecting them
⊙Population
○ All people, animals, or objects that have
at least one characteristic in common;
Ex: all undergraduate students
⊙Sample of subjects
○ A group that is a subset of the
population of interest
⊙Representativeness
○ How closely the sample mirrors the
large population
49

50. Probability
Sampling
⊙Selecting subjects in such a way that
the odds of their being in the study are
known or can be calculated;
○ Begin by defining the sample you
want to study (ex: women born in
1975 now living in Seattle), then
choose an unbiased method for
selecting the subjects (random
selection: any member of the
population has an equal opportunity
to be selected)
50

51. ⊙Simple Random Sampling
○ Most basic form of probability sampling; a
portion of the whole population is selected
in an unbiased way; all members of the
population being studied must have an
equal chance of being selected
⊙ Systematic Random Sampling
○ All members of the population are known
and can be listed in an unbiased way; a
research picks the nth person; n is
determined by size of population and the
desired sample size
51

52. ⊙Stratified Random Sampling
○ Used when populations have distinct
subgroups; obtained by randomly
sampling from people in each
subgroup in the same proportions
as they exist in the population
○ Example: majors at Clemson; 50% of
the students are engineering majors
■ Our sample of the population has
to have 50% engineers to
represent the general population
52

53. ⊙Cluster Sampling
○ Sample entire clusters or naturally
occurring groups that exist within the
population; used if individual sampling is
impossible due to cost or too large of a
population; less reliable (example of
clusters: zip code areas, school districts,
etc)
○ Example: If you want to sample students
from the business of behavioral science,
you just give everyone who takes intro to
psych a survey
53

54. Nonprobability
Sampling
⊙Subjects are not
chosen at random
54

55. ⊙Quota Sampling
○ Select samples through
predetermined quotas that
reflect the makeup of the
population
⊙Convenience Sampling
○ Using any groups who happen
to be available
55

56. ⊙Purposive Sampling
○ When nonrandom samples are selected
because the individuals reflect a
specific purpose of the study
○ Ex: Comparing new training program for
employees in two departments
■ Select the employees of those two
departments
⊙Snowball Sampling
○ Researcher locates one or a few people
who fit the criteria and asks these
people to find more people
56

57. ⊙Reporting Samples
○ The way a sample is chosen
influences what can be concluded
from the result
○ Must explain the type of sample
used & how subjects were recruited
○ Details that may have influenced the
type of subject need to be reported
(ex: if they got paid, fulfilled a course
requirement, etc)
57

58. 58