This document provides an overview of various statistical analyses that can be used in psychological research, including t-tests, analysis of variance (ANOVA), repeated measures ANOVA, factorial ANOVA, mixed ANOVA, analysis of covariance (ANCOVA), and multivariate analysis of variance (MANOVA). Examples of research questions that could be addressed with each type of analysis are provided. Key assumptions, procedures, and interpretations for each analysis are discussed at a high level.

1. Lecture 10 Survey Research & Design in Psychology James Neill, 2011 ANOVA II

3. Independent samples t -test

4. Paired samples t -test

5. One-way ANOVA

6. One-way repeated measures ANOVA

7. Factorial ANOVA

8. Mixed design ANOVA

9. ANCOVA

10. MANOVA

12. Ch12 Multiple Comparisons Among Treatment Means

13. Ch13 Factorial Analysis of Variance

14. Ch14 Repeated-Measures Designs

15. Ch16 Analyses of Variance and Covariance as General Linear Models Readings See also: Inferential statistics decision-making tree

17. Difference between two independent groups -> independent samples t -test = BETWEEN-SUBJECTS

18. Difference between two related measures (e.g., repeated over time or two related measures at one time) -> paired samples t -test = WITHIN-SUBJECTS Previous lecture

19. Are the differences in a sample generalisable to a population?

21. Overall variance is partitioned into between-group and within-group variance

23. 1 or more IVs: categorical

25. 1-way repeated measures ANOVA

26. Factorial ANOVA

27. Mixed ANOVA

28. ANCOVA

29. MANOVA

31. 1-way repeated measures ANOVA

32. Factorial ANOVA

33. Mixed ANOVA

34. ANCOVA

35. MANOVA

37. 1-way repeated measures ANOVA

38. Factorial ANOVA

39. Mixed ANOVA

40. ANCOVA

41. MANOVA

43. 1-way repeated measures ANOVA

44. Factorial ANOVA

45. Mixed ANOVA

46. ANCOVA

47. MANOVA

49. 1-way repeated measures ANOVA

50. Factorial ANOVA

51. Mixed ANOVA

52. ANCOVA

53. MANOVA

55. 1-way repeated measures ANOVA

56. Factorial ANOVA

57. Mixed ANOVA

58. ANCOVA

59. MANOVA

61. Follow-up tests : Which of the M s differ?

62. Effect size : How large are the observed differences?

65. p = probability that the observed mean differences between groups could be attributable to chance

66. F is the ratio of between-group : within-group variance

69. post-hoc contrasts .

70. One-way ANOVA

72. Normality : Normally distributed for all IV groups (robust to violations of this assumption if N s are large and approximately equal e.g., >15 cases per group)

73. Variance : Equal variance across for all IV groups (homogeneity of variance)

74. Independence : Participants' data should be independent of others' data

75. One-way ANOVA: Are there differences in satisfaction levels between students who get different grades ?

76. Recoding needed to achieve min. 15 per group.

77. These groups could be combined.

78. The recoded data has more similar group sizes and is appropriate for ANOVA.

79. SD s are similar (homogeneity of variance). M s suggest that higher grade groups are more satisfied.

80. Levene's test indicates homogeneity of variance.

81. Follow-up tests should then be conducted because the effect of Grade is statistically significant ( p < .05).

83. 40-45 year olds

85. Higher = external

86. 60-65 year-olds appear to be more internal, but the overlapping confidence intervals indicate that this may not be statistically significant.

87. The SD s vary between groups (the third group has almost double the SD of the younger group). Levene's test is significant (variances are not homogenous).

88. There is a significant effect for Age ( F (2, 59) = 4.18, p = .02). In other words, the three age groups are unlikely to be drawn from a population with the same central tendency for LOC.

89. Which age groups differ in their mean locus of control scores? (Post hoc tests). Conclude: Gps 0 differs from 2; 1 differs from 2

91. Planned : Compares specific combinations (Do one or the other; not both)

93. Scheffe, Bonferroni, Tukey’s HSD, or Student-Newman-Keuls

94. Keeps experiment-wise error rate to a fixed limit

96. Specify contrast coefficients to weight the comparisons (e.g., 1 st two vs. last one)

97. Tests each contrast at critical 

99. Test using Mauchly's test of sphericity (If Mauchly’s W Statistic is p < .05 then assumption of sphericity is violated.)

101. The obtained F ratio must then be evaluated against new degrees of freedom calculated from the Greenhouse-Geisser , or Huynh-Feld, Epsilon values.

103. 6 months

104. 12 months

105. Mean LOC scores (with 95% C.I.s) across 3 measurement occasions Not much variation between means.

106. Descriptive statistics Not much variation between means.

107. Mauchly's test of sphericity Mauchly's test is not significant, therefore sphericity can be assumed.

108. Tests of within-subject effects Conclude: Observed differences in means could have occurred by chance ( F (2, 118) = 2.79, p = .06) if critical alpha = .05

109. 1-way repeated measures ANOVA Do satisfaction levels vary between Education, Teaching, Social and Campus aspects of university life?

112. Tests of within-subject effects

113. Factorial ANOVA (2-way): Are there differences in satisfaction levels between gender and age ?

117. Main effect of IV2

119. there are more than 2 levels of an IV are involved then follow-up tests are required. Factorial ANOVA

125. Factorial ANOVA (2-way): Are there differences in LOC between gender and age ?

126. Example: Factorial ANOVA Main effect 1 : - Do LOC scores differ by Age? Main effect 2 : - Do LOC scores differ by Gender? Interaction : - Is the relationship between Age and LOC moderated by Gender? (Does any relationship between Age and LOC vary as a function of Gender?)

130. Plot of LOC by Age and Gender

131. Age x gender interaction

132. Error-bar graph for Age main effect Age main effect

133. Descriptives for Age main effect

134. Error-bar graph for Gender main effect Gender main effect

135. Descriptives for Gender main effect

136. Descriptives for LOC by Age and Gender

137. Tests of between-subjects effects

139. IV2 = Light and heavy exercise

140. DV = Av. hours of sleep per night Interactions

141. Interactions

142. Interactions

144. Since such experiments have mixtures of between subjects and within-subject factors they are said to be of mixed design

146. IV2 is within-subjects (e.g., Social Satisfaction and Campus Satisfaction)

148. Main effect of IV2

150. Post-hoc contrasts Mixed design ANOVA (SPANOVA)

152. Within-subjects = Spacing (Narrow, Medium, Wide)

153. Gender - varies within-subjects Mixed design ANOVA (SPANOVA)

155. Brackets signify a mixed design with repeated measures on Factor B

157. Homogeneity of variance

158. Sphericity

159. Homogeneity of inter-correlations

161. The assumption is tested using Box’s M statistic

162. Homogeneity is present when the M statistic is NOT significant at p > .001.

163. Mixed design ANOVA: Example Do satisfaction levels vary between gender for education and teaching ?

165. Tests of within-subjects contrasts

166. Tests of between-subjects effects

169. Extension of ANOVA, using ‘regression’ principles

171. another variable (DV)

172. after controlling for a third variable (CV)

174. Effect of the covariate on the DV is removed (or partialled out) (akin to Hierarchical MLR)

178. Increases power of F -test

179. May not be able to achieve experimental control over a variable (e.g., randomisation), but can measure it and statistically control for its effect.

181. The differences between P s on the CV are removed, allowing focus on remaining variation in the DV due to the IV.

182. Make sure hypothesis (hypotheses) is/are clear.

184. Normality

185. Homogeneity of Variance (use Levene’s test)

187. Independence of IV and CV.

188. Multicollinearity - if more than one CV, they should not be highly correlated - eliminate highly correlated CVs.

189. Reliability of CVs - not measured with error - only use reliable CVs.

191. If the CV is not correlated with the DV there is no point in using it.

193. Check via scatterplot with lines of best fit.

194. Assumptions of ANCOVA

195. ANCOVA example 1: Does education satisfaction differ between people with different levels of coping (‘Not coping’, ‘Just coping’ and ‘Coping well’) with average grade as a covariate?

201. DV = academic achievement

202. CV = motivation

203. Experimental design - assume students randomly allocated to different teaching methods.

204. ANCOVA example Academic Achievement (DV) Teaching Method (IV) Motivation (CV)

205. ANCOVA example Academic Achievement Teaching Method Motivation

208. F has gone from 1 to 5 and is significant because the error term (unexplained variance) was reduced by including motivation as a CV. ANCOVA example 2

210. ANCOVA more commonly used if IV is categorical.

212. ANCOVA allows us to statistically control for one or more covariates.

215. homogeneity of variance (Levene’s test)

216. Linearity between CV & DV (scatterplot)

218. MANOVA Multivariate Analysis of Variance

220. MANOVA tests whether mean differences among groups on a combination of DV’s are likely to have occurred by chance.

222. One-way repeated measures ANOVA and mixed ANOVA are subsets of MANOVA.

223. If multivariate F is significant, then examine univariate F s MANOVA

225. MANOVA provides separate ANOVAs as well as a multivariate ANOVA. MANOVA

231. e.g., Desensitisation may have an advantage over relaxation training or control, but only on test anxiety. The effect is missing if anxiety is not one of your DV’s. MANOVA advantages over ANOVA

233. When responses to two or more DV’s are considered in combination, group differences become apparent. MANOVA advantages over ANOVA

235. When cell sizes > 30 assumptions of normality and equal variances are of little concern.

236. Equal cell sizes preferred (but not essential)

237. Ratios of smallest to largest cell sizes of 1:1.5+ may cause problems. MANOVA

239. Multivariate outliers which affect normality can normally be identified using Mahalanobis distance in the Regression sub-menu. MANOVA assumptions

241. Homogeneity of regression : It is assumed that the relationships between covariates and DV’s in one group is the same as other groups. Necessary if stepdown analyses required. MANOVA assumptions

243. Box’s M test is used for this assumption and should be non-significant p >.001.

244. Multicollinearity and singularity : When correlations among DV’s are high, problems of multicollinearity exist. MANOVA assumptions

246. Wilks’ lambda is one such statistic.

248. Partial eta-square (  p 2 ) provides an estimate of the effects for each IV.

249. Cohen’s d : Standardised differences between two means. Effect sizes

251. = SS between / SS total = SS B / SS T

252. = prop. of variance in Y explained by X

253. = Non-linear correlation coefficient

254. = prop. of variance in Y explained by X

255. Ranges between 0 and 1

258. .06 medium

259. .14 large

260.  2 = SS between /SS total = 395.433 / 3092.983 = 0.128 Eta-squared is expressed as a percentage: 12.8% of the total variance in control is explained by differences in Age

262.  2 is not provided by PASW – calculate separately.

263. R 2 at the bottom of PASW F -tables is the linear effect as per MLR – however, if an IV has 3 or more non-interval levels, this won’t equate with  2 .

265. Test the assumptions , esp. LOM, normality, univariate and multivariate outliers, homogeneity of variance, N

266. Present the descriptive statistics (usually in a table)

267. Consider presenting a figure to illustrate the data (bar, error-bar or line graph)

269. Conduct planned or post-hoc testing as appropriate

270. Report and interpret effect sizes

271. State whether or not results support hypothesis (hypotheses)

273. F is an omnibus “gateway” test; may require follow-up tests.

274. Conduct follow-up tests where sig. Main effects have three ore more levels.

276. Repeated measure designs include the assumption of sphericity

278. Partial eta-square (  p 2 )for each IV

279. Standardised mean differences for the differences between each pair of means

281. Free and open source software.

282. http://www.openoffice.org/product/impress.html