3. Randomized Controlled Experiment
Pre-test Post-test
R Experimental Group Mexp1 x Mexp2
R Control Group Mcontrol1 Mcontrol2
X = experimental manipulation
M = measurements
Experimental effect = (Mexp2 - Mexp1)- (Mcontrol2 - Mcontrol1)
R = Random assignment of subjects to groups
4. designs and validity
• last time: validity of measures
and indicators
• this time: internal & external
validity as aspects of research
design
5. internal validity XZ
• extent to which a research design yields strong evidence of causality
• Key issue: is there a causal relationship between a “treatment” and the
“outcome(s)”?
Threats to internal validity:
1. History
2. Maturation
3. Testing
4. Instrumentation
5. Statistical Regression
6. Selection Threat
7. Experimental Mortality
8. Selection-Maturation Interaction
6. external validity XZ
• extent to which a research design yields findings that generalize to
• To subjects / objects / units other than those actually in your study
• To settings / circumstances other than that/those actually in your study
• To times other than those actually in your study
Threats to external validity:
1. Interaction of testing and experimental variable
2. Interaction of selection bias and experimental variable
3. Reactive Arrangements
4. Multiple-treatment interference
7.
8. How might you design an experiment to test a new
drug?
9. How might you design an experiment to test a new
drug?
15. Posner
• Why are the Chekwas and
Tumbukas allies in Zambia and
adversaries in Malawi?
• unit of analysis (530)
• case selection criteria (531)
• survey design (532)
• lit review (535)
• additional tests (541)
• conclusions (543)
• Posner’s approach: find the
treatment
17. group activity
1. Characterize your project,
or a partner’s project, in
terms of its research design
2. What are some of the
potential threats to internal
and external validity?