2. The results of a research study are only useful
to the extent that they can be accurately and
confidently interpreted.
The issue of accurate and confident
interpretation of results is at the center of any
discussion of validity.
3. From validus, meaning “strong”
Refers to the degree with which correct
inferences can be made from the results of a
research study.
5. Refers to the extent to which the results
obtained in a research study are a function of
the variables that were systematically
manipulated, measured, and /or observed in
the study.
6. For example, a researcher in interested in
determining which of two instructional
methods is superior for teaching a history
concept. Suppose further that the researcher
asked two teachers to each use one of the
methods of instruction and then compared the
mean test scores of each class following the
instructions.
7. Teachers’ teaching effectiveness and
enthusiasm
Class interest or preparation
One class may have interruptions (fire drills)
8.
9. Refers to the occurrence of events that could
alter the outcome or the results of a study.
10. Refers to the occurrence of events that could
alter the outcome or the results of a study.
Previous history – occurs before the study
A study of the effectiveness of a new method for
teaching a unit on the biology of a cell, many watched a
television documentary about “The Cell”
11. Refers to the occurrence of events that could
alter the outcome or the results of a study.
Concurrent history – occurs during the study
Eg. Studying the effectiveness of using musical
activities to teach mathematics concepts. While one
teacher uses the standard curriculum, another teacher is
using the musical activities curriculum
12. Pertains to any changes that occur in the
subjects during the course of the study that are
not part of the study and that might affect the
results of the study.
13. Pertains to any changes that occur in the
subjects during the course of the study that are
not part of the study and that might affect the
results of the study.
Biological ( growth processes)
Eg. Weight gain or increase in height due to breakfast
or lunch program
14. Pertains to any changes that occur in the
subjects during the course of the study that are
not part of the study and that might affect the
results of the study.
Psychological (learning or development)
Eg. Effects of certain instructional techniques on
concept learning of sixth graders, attainment of certain
operational thought during that period has to be
considered
15. Relates to the possible effects of a pretest on the
performance of participants in a study on the
posttest.
May alert subjects to the fact that they are being
studied
May affect performance on later administrations
16. Is concerned with the effects on the outcome of
a study of the inconsistent use of a
measurement instrument (what the instrument
is measuring changes during the duration of
the study).
Eg. The effects of fatigue on an achievement
test
17. Refers to the tendency of extreme scores to
move (or regress) toward the mean score on
subsequent retesting.
Eg. Students scoring below 25% (lowest extreme) on
an IQ test are given a posttest. A higher posttest
score is expected.
18. Refers to the loss of subjects from a study due
to their initial nonavailability or subsequent
withdrawal from the study.
Eg. More high-scoring people drop out from the
experimental group than from the control group
19. Pertains to the possibility that groups in a
study may possess different characteristics and
that those differences may affect the results.
Differences in age, ability, gender or racial/ethnic
composition, or any of an almost unlimited number
of ways.
20.
21. 1. Standardization of the conditions under
which the research study is carried out will
help minimize threats to internal validity from
history and instrumentation.
22. 2. Obtaining as much information as possible
about the participants in the research study
aids in minimizing threats to internal validity
from mortality and selection.
23. 3. Obtaining as much information as possible
about the procedural details of the research
study, for example, where and when the
study occurs, minimizes threats to internal
validity from history and instrumentation.
24. 4. Choosing an appropriate research design can
help control most other threats to internal
validity.
25.
26. The use of a control group, selected from the
same population as the experimental group(s)
and which experiences the same concurrent
history as the experimental group(s), can help
eliminate most of the effects of history. Also,
the shorter the duration of an experiment, the
less likely history will be a threat.
27. The effects of maturation, like the effects of
history, can be minimized by the use of a
control group, selected from the same
population as the experimental group(s). Also,
like the effects of history, the effects of
maturation tend to be minimized in studies of
short duration.
28. The use of a research design that does not
include a pretest can eliminate testing as a
potential threat to internal validity. If baseline
or pretreatment or data are needed, the use of
unobtrusive measures (data collection
techniques about which the experimental
participant is unaware) may minimize the
effects of testing. It also may help for a
researcher to use different equivalent forms of
a test for pretesting and posttesting.
29. Careful specification and control of the
measurement procedures can eliminate most
instrumentation threats. Standardized
instruments, administration or data collection
procedures, and the training or observers are
among the procedures that help control the
instrumentation threat.
30. Avoiding the use of extreme scorers, when
average scorers are excluded, will minimize the
threat.
31. Choosing large groups and ensuring that they
are representative of the population from
which they were selected can minimize
mortality threats. The use of follow-up
procedures with a portion of those who leave
the study or who were initially unavailable can
further minimize mortality as a threat
32. Random selection and random assignment of
subjects minimize selection as a threat to
internal validity. If random selection and
assignment are not possible, the use of certain
statistical techniques, used as part of a careful
quasi-experimental design, can adjust for
group differences and thereby minimize
selection as a threat.
33.
34. Refers to the extent to which the results of a
research study are able to be generalized
confidently to a group larger than the group
that participated in the study. (Bracht & Glass)
35. Refers to the extent to which the results of a
research study are able to be generalized
confidently to a group larger than the group
that participated in the study. (Bracht & Glass)
The researcher must have reason to believe that
the variables used in the study are similar to
those aspects as they exist in the larger
population.
36. The extent to which a sample is representative
(or not representative) of the population from
which it was selected.
37. The extent to which characteristics of the
setting or context of the research study are
representative (or not representative ) of the
setting and context to which the results are to
be generalized.
38.
39. Refers to the fact that the administration of a
test (for example, a pretest) may affect the
responses or the performance of the
participants in a research study.
Its results may not be generalizable to
situations where pretesting will not occur.
40. It pertains to the situation in which participants
in a study receive more than one treatment. In
such a case, the effects of the multiple
treatment may interact.
41. It is concerned with the possibility that some
characteristic of the participants selected for the
study interacts with some aspect of the
treatment.
May include prior experiences, learning, personality
factors, or any traits that might interact with the
effect of the treatment
42. It pertains to situations where participants
become aware that they are involved in a
study, and , as a result of that awareness, their
response or performance is different from what
it would have been otherwise.
43. It pertains to situations where participants
become aware that they are involved in a
study, and , as a result of that awareness, their
response or performance is different from what
it would have been otherwise.
“Novelty effect” (newness) or belief on the part of the
participants that they are receiving some “special”
treatment (sometimes known as Hawthorne Effect.)
“Placebo effect” or to the participants’ belief in the
effectivenesss of the treatment
44. They refer to the possibility that an
experimenter may sometimes unintentionally
influence the performance of participants in a
study.
45. They refer to the possibility that an
experimenter may sometimes unintentionally
influence the performance of participants in a
study.
46. They refer to the possibility that an
experimenter may sometimes unintentionally
influence the performance of participants in a
study.
PASSIVE
Gender, race, or personal attributes of the researcher
47. They refer to the possibility that an
experimenter may sometimes unintentionally
influence the performance of participants in a
study.
PASSIVE
Gender, race, or personal attributes of the researcher
ACTIVE
The expectations of the researcher or observer are
communicated to the participant in a manner that affects
performance
48. It is concerned with the extent to which the
variables in a study are adequately described
and operationally defined. Variables can be
defined too specifically.
49.
50. Use research designs that do not include
pretests.
Solomon four-group design is useful in
determining the extent to which pretesting
may have influenced the results of the study.
51. The researcher should try to choose a design in
which only one treatment is assigned to each
subject. If such a design is practical, the
researcher should try to control and/or
measure the effects of all relevant treatments
and incorporate them into a multiple-treatment
design.
52. Use random selection and assignment of
participants.
If the above is not practical, use statistical
techniques such as analysis of covariance, used
in conjunction with a quasi-experimental
design, can take into account differences due to
measurable attributes of the individual.
53. Have a control group and a placebo group
Both should be treated the same in all respects,
except in regard to the treatment itself
All of the participants should believe they are
being treated comparably
54. Use “blind” data collection procedures. This
means that the researcher does not collect data
or make observations but instead trains a
“naïve” observer to do so.
The person collecting the data or making the
observations should be unaware of the purpose
of the study and should be unaware of which
participants are receiving the experimental
treatment.
55. Must operationally define variables carefully in
a way that is meaningful in settings beyond
that in which the study is being conducted.
Use of widely agreed upon definitions or
multiple competing definitions should be
considered.
