experimental control

1. Experimental Control
Psych 231: Research
Methods in Psychology

2. Colors and words
 Divide into two groups:
 left side of room
 right side of room
 Instructions: Read aloud the COLOR that the words are
presented in. When done raise your hand.
 Left side first. Right side people please close your
eyes.
 Okay ready?

3. Blue
Green
Red
Purple
Yellow
Green
Purple
Blue
Red
Yellow
Blue
Red
Green
List 1

4.  Okay, now it is the right side’s turn.
 Remember the instructions: Read aloud the
COLOR that the words are presented in. When
done raise your hand.
 Okay ready?

5. Blue
Green
Red
Purple
Yellow
Green
Purple
Blue
Red
Yellow
Blue
Red
Green
List 2

6. Our results
 So why the difference between the results for
the people on the right side of the room versus
the left side of the room?
 Is this support for a theory that proposes:
 “good color identifiers usually sit on the left side of a
room”
 Why or why not? Let’s look at the two lists.

7. Blue
Green
Red
Purple
Yellow
Green
Purple
Blue
Red
Yellow
Blue
Red
Green
List 2
Right
side
Blue
Green
Red
Purple
Yellow
Green
Purple
Blue
Red
Yellow
Blue
Red
Green
List 1
Left side
Matched Mis-Matched

8.  What resulted in the perfomance
difference?
 Our manipulated independent variable
 The other variable match/mis-match?
 Because the two variables are
perfectly correlated we can’t tell
 This is the problem with confounds
Blue
Green
Red
Purple
Yellow
Green
Purple
Blue
Red
Yellow
Blue
Red
Green
Blue
Green
Red
Purple
Yellow
Green
Purple
Blue
Red
Yellow
Blue
Red
Green

9. Experimental Control
 Our goal:
 To test the possibility of a
relationship between the
variability in our IV and how
that affects the variability of
our DV.
• Control is used to minimize
excessive variability.
• To reduce the potential of
confounds.

10. Sources of variability (noise)
Nonrandom (NR) Variability - systematic variation
A. (NRexp) manipulated independent variables (IV)
i. our hypothesis is that changes in the IV will result in
changes in the DV
 Sources of Total (T) Variability:
T = NRexp + NRother + R

11. Sources of variability (noise)
Nonrandom (NR) Variability - systematic variation
B. (NRother) extraneous variables (EV) which covary with IV
i.other variables that also vary along with the changes in the
IV, which may in turn influence changes in the DV
(Condfounds)
 Sources of Total (T) Variability:
T = NRexp + NRother + R

12. Sources of variability (noise)
Non-systematic variation
C. Random (R) Variability
• Imprecision in manipulation (IV) and/or measurement (DV)
• Randomly varying extraneous variables (EV)
 Sources of Total (T) Variability:
T = NRexp + NRother + R

13. Sources of variability (noise)
 Sources of Total (T) Variability:
T = NRexp + NRother + R
Goal: to reduce R and NRother so that we can detect NRexp.
That is, so we can see the changes in the DV that are due to the
changes in the independent variable(s).

14. Weight analogy
 Imagine the different sources of variability as
weights
R
NR
exp
NR
other
R
NR
other
Treatment group control group
The effect of the treatment

15. Weight analogy
 If NRother and R are large relative to NRexp
then detecting a difference may be difficult
R
NR
exp
NR
other
R
NR
other
Difference
Detector

16. Weight analogy
 But if we reduce the size of NRother and R
relative to NRexp then detecting gets easier
R
NR
other
R
NR
exp
NR
other
Difference
Detector

17. Things making detection difficult
 Potential Problems
 Excessive random variability
 Confounding
 Dissimulation

18. Potential Problems
 Excessive random variability
 If control procedures are not applied
• then R component of data will be excessively large, and
may make NR undetectable
 So try to minimize this by using good measures of
DV, good manipulations of IV, etc.

19. Excessive random variability
R
NR
exp
NR
other
NR
other
R
Hard to detect the effect of NRexp
Difference
Detector

20. Potential Problems
 Confounding
 If relevant EV co-varies with IV, then NR
component of data will be "significantly" large, and
may lead to misattribution of effect to IV
IV
DV
EV
Co-vary together

21. Confounding
R
NR
exp
NR
other
Hard to detect the effect of NRexp because the
effect looks like it could be from NRexp but is
really (mostly) due to the NRother
R
Difference
Detector

22. Potential Problems
 Potential problem caused by experimental
control
 Dissimulation
• If EV which interacts with IV is held constant, then effect of
IV is known only for that level of EV, and may lead to
overgeneralization of IV effect
 This is a potential problem that affects the
external validity

23. Controlling Variability
 Methods of Experimental Control
 Comparison
 Production
 Constancy/Randomization

24. Methods of Controlling Variability
 Comparison
 An experiment always makes a comparison, so it must have at
least two groups
• Sometimes there are control groups
• This is typically the absence of the treatment
• Without control groups if is harder to see what is really
happening in the experiment
• It is easier to be swayed by plausibility or inappropriate
comparisons
• Sometimes there are just a range of values of the IV

25. Methods of Controlling Variability
 Production
 The experimenter selects the specific values of the Independent
Variables
• Need to do this carefully
• Suppose that you don’t find a difference in the DV across your
different groups
• Is this because the IV and DV aren’t related?
• Or is it because your levels of IV weren’t different enough

26. Methods of Controlling Variability
 Constancy/Randomization
 If there is a variable that may be related to the DV that you
can’t (or don’t want to) manipulate
• Control variable: hold it constant
• Random variable: let it vary randomly across all of the
experimental conditions
 But beware confounds, variables that are related to both the
IV and DV but aren’t controlled

27. Experimental designs
 So far we’ve covered a lot of the about details
experiments generally
 Now let’s consider some specific experimental
designs.
 Some bad designs
 Some good designs
• 1 Factor, two levels
• 1 Factor, multi-levels
• Factorial (more than 1 factor)
• Between & within factors

28. Poorly designed experiments
 Example: Does standing close to somebody cause
them to move?
 So you stand closely to people and see how long before they
move
 Problem: no control group to establish the comparison group
(this design is sometimes called “one-shot case study
design”)

29. Poorly designed experiments
 Does a relaxation program decrease the urge to
smoke?
 One group pretest-posttest design
 Pretest desire level – give relaxation program – posttest desire
to smoke

30. Poorly designed experiments
 One group pretest-posttest design
 Problems include: history, maturation, testing, instrument
decay, statistical regression, and more
participants Pre-test Training
group
Post-test
Measure
Independent
Variable
Dependent
Variable
Dependent
Variable

31. Poorly designed experiments
 Example: Smoking example again, but with two groups.
The subjects get to choose which group (relaxation or
no program) to be in
 Non-equivalent control groups
 Problem: selection bias for the two groups, need to do random
assignment to groups

32. Poorly designed experiments
 Non-equivalent control groups
participants
Training
group
No training
(Control) group
Measure
Measure
Self
Assignment
Independent
Variable
Dependent
Variable

33. “Well designed” experiments
 Post-test only designs
participants
Experimental
group
Control
group
Measure
Measure
Random
Assignment
Independent
Variable
Dependent
Variable

34. “Well designed” experiments
 Pretest-posttest design
participants
Experimental
group
Control
group
Measure
Measure
Random
Assignment
Independent
Variable
Dependent
Variable
Measure
Measure
Dependent
Variable