CauseandComparative AND Experimental

1. Educational Research: Causal-Comparative & Experimental Studies ELT-718 Research Methods Asst. Prof. Dr. Hasan BEDİR

3. The basic steps of research... Scientific and disciplined inquiry is an orderly process, involving:  description and execution of procedures to collection information (“ method ”)  objective data analysis  statement of findings (“ results ”)  recognition and identification of a topic to be studied (“ problem ”)

4. Research methods... Quantitative … … collects and analyzes numerical data obtained from formal instruments

8. Causal-Comparative Research The Purpose Purpose of explaining educational phenomena through the study of cause-and-effect relationships. The presumed cause is called the independent variable and the presumed effect is called the dependent variable . Designs where the researcher does not manipulate the independent variable are called ex post facto research .

9. Causal-Comparative Research ( Continued ) Causal-Comparative research is also a type of non-experimental investigation in which researchers seek to identify cause-effect relationships by forming groups of individuals in whom the independent variable is present or absent and than determining whether the groups differ on the dependent variable.

11. Research variables... Independent … … an activity of characteristic believed to make a difference with respect to some behavior … (syn.) experimental variable, cause, treatment

12. dependent variable … … the change or difference occurring as a result of the independent variable … (syn.) criterion variable, effect, outcome, posttest

13. Data analysis and interpretation… … researcher uses a variety of descriptive and inferential statistics : mean standard deviation t-test analysis of variance chi squared

23. Types of experimental comparison… 1. comparison of two different approaches ( A versus B ) 2. comparison of an existing approach to a new approach ( A and ~ A ) 3. comparison of differing amounts of a single approach ( A and a or a and A )

24. where: A – experimental (“treatment”) group B – control (“no treatment,” “nonmanipulated”) group

25. Group experimental designs… 1. single-variable 2. factorial

27. one-group pretest-posttest design O X O … a single group is pretested ( O ), exposed to a treatment ( X ) and, then, is posttested ( O )

28. static group comparison X 1 O X 2 O … involves at least two groups ( X ), one receiving a new, or experimental treatment ( X 1 ) and another receiving a traditional, or control treatment ( X 2 ) and, then, are posttested ( O )

31. … at least two groups are formed by random assignment ( R ), administered a pretest ( O ), receive different treatments ( X 1 , X 2 ), are administered a posttest, and posttest scores are compared to determine effectiveness of treatments

32. posttest-only control group design R X 1 O R X 2 O

33. … at least two groups are formed by random assignment ( R ), receive different treatments ( X 1 , X 2 ), are administered a posttest, and posttest scores are compared to determine effectiveness of treatments

34. Solomon four-group design R O X 1 O R O X 2 O R X 1 O R X 2 O

35. … four groups are formed by random assignment ( R ) of participants, two groups are pretested ( O ) and two are not, one pretested and one unpretested group receive the experimental treatments ( X 1 , X 2 ), each group is are administered a posttest on the dependent variable, and posttest scores are compared to determine effectiveness of treatments

37. Experimental Design Factorial Design Diagram Independent Variable #1: Teaching Method Independent Variable #2: Aptitude Randomly assigned 3 rd graders scoring below 60 on an aptitude test. Randomly assigned 3 rd graders scoring below 60 on an aptitude test. Randomly assigned 3 rd graders scoring about 85 on an aptitude test. Randomly assigned 3 rd graders scoring above 85 on an aptitude test. Reading/Lecture/Etc. Lecture Only High Low

38. Independent Variable #1 Teaching Method How many possible Teaching Methods are there? Which will be the methods used in the study? If more than one will be used, each method may be considered a factor of the variable known as Teaching Method. Teaching Method Lecture only Lecture & Small Group Discussion

39. Independent Variable #2 Aptitude How many possible levels of aptitude are there? How many may be represented in the group of subjects participating in the study? Once identified, levels of aptitude may be considered factors of the variable known as Aptitude. Low High Aptitude

40. Lecture only Lecture & Small Group Discussion Low High

42. A 2 X 2 factorial design… Independent Variable A B Dependent Variable O O Group #1 Group #2 Group #3 Group #4 Cells not manipulated manipulated

43. A 2 X 2 factorial design… A No interaction between factors B

44. A 2 X 2 factorial design… A No interaction between factors B

45. A 2 X 2 factorial design… A Interacting factors B

46. A 2 X 2 factorial design… A Interacting factors B

47. two-by-three factorial design (six cells) 2 X 3 … two types of factors (e.g., motivation; interest) each of which has three levels (e.g., high, medium, low)

48. Single-subject experimental designs… 1. A – B – A withdrawal 2. multiple baseline designs 3. alternating treatments designs

54. …“ multiple” refers to the study of more than one behavior, participant, or setting

55. … instead of collecting baseline data on one specific behavior, data are collected on: (1) several behaviors for one participant, (2) one behavior for several participants, or (3) one behavior and one participant in several settings

56. … then, over a period of time, the treatment is systematically applied to each behavior (or participant, or setting) one at a time until all behaviors (or participants or settings) have been exposed to the treatment

58. Threats to validity… … internal : factors other than the independent variable that affect the dependent variable … external : factors that affect the generalizability of the study to groups and settings beyond those of the experiment

59. Threats to internal validity… 1. history 2. maturation 3. testing 4. instrumentation 5. statistical regression 6. differential selection of participants 7. mortality 8. selection-maturation interaction

68. Threats to external validity… 1. pretest-treatment interaction 2. selection-treatment interaction 3. multiple treatment interference 4. specificity of variables 5. treatment diffusion 6. experimenter effects 7. reactive effects

76. Types of reactive arrangements… … Hawthorne effect : any situation in which participants’ behavior is affected not by the treatment per se but by their knowledge of participating in a study … compensatory rivalry : the control group is informed that they will be the control group for a new, experimental study (“ John Henry effect ”)

77. … placebo effect : the situation in which half of the participants receive no treatment but believe they are … novelty effect : the situation in which participant interest, motivation, or engagement increases simply because they are doing something different

78. Controlling for extraneous (confounding) variables… 1. randomization 2. matching 3. comparing homogeneous groups or subgroups 4. using participants as their own controls 5. analysis of covariance (ANCOVA)

85. … causal-comparative studies typically involve two (or more) groups and one independent variable, whereas correlational studies typically involve two (or more) variables and one group True

86. … causal-comparative studies involve relation, whereas correlational studies involve cause False

87. … oftentimes, causal-comparative research is undertaken because the independent variable could be manipulated but should not True

88. … one of the most important reasons for conducting causal-comparative research is to identify variables worthy of experimental investigation True

89. …“ lack of control” means that the researcher can and should manipulate the independent variable False

90. … each group in a causal-comparative study represents a different population True

91. … the more similar two groups are on all relevant variables except the independent variable, the stronger the study is True

92. … there is random assignment to treatment groups from a single population in causal-comparative studies False

93. … lack of randomization, manipulation of the independent variable, and control are all sources of weakness in a causal-comparative design True

94. … matching, comparing homogenous groups or subgroups, and covariate analysis are strategies that enable researchers to overcome problems of initial group differences on an extraneous variable True

95. … interpretation of the findings in a causal-comparative study requires considerable caution because the cause may be the effect and the effect may be the cause True

96. … extraneous variables or confounding factors may be the real “cause” of both the independent and dependent variables True