Sreb March 2010 5
•Télécharger en tant que PPT, PDF•
0 j'aime•419 vues
Presentation on large-scale e-Learning for Educators online professional development program and research with online training and courses by EdTech Leaders Online at EDC.
![Today’s Plan ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à Sreb March 2010 5
Similaire à Sreb March 2010 5 (20)
Plus de Barbara Treacy
Plus de Barbara Treacy (8)
Dernier
Dernier (20)
Sreb March 2010 5
- 1. New Evidence: Proving Online PD Increases Teacher Performance Barbara Treacy Director, EdTech Leaders Online (ETLO) Education Development Center (EDC)
- 19. 4 th Grade Teacher Results- Knowledge (Sample) Instrument Experimental Group (n = 36) Control Group (n = 49) ELA Pre 37% 36% Post 52% 38% Vocabulary Pre 16% 14 % Post 22% 14 % Reading Comprehension Pre 47% 46% Post 65% 49% Writing Pre 49% 49% Post 65% 47%
- 25. Research Data and Procedures
- 30. 4 TH GRADE RESULTS
- 31. 4 th Grade Teacher Instruments Instrument Number of Items Pre-Test Reliability Post-Test Reliability Knowledge Total ELA 21 .805 .833 Vocabulary 8 .820 .862 Reading Comprehension 7 .658 .722 Writing 6 .511 .497 Practice Vocabulary 16 .887 .904 Reading Comprehension 16 .879 .879 Writing 14 .911 .915
- 32. 4 th Grade Teacher Results- Knowledge Instrument Experimental Group (n = 36) Control Group (n = 49) ELA Pre 37% 36% Post 52% 38% Vocabulary Pre 16% 14 % Post 22% 14 % Reading Comprehension Pre 47% 46% Post 65% 49% Writing Pre 49% 49% Post 65% 47%
- 33. 4 th Grade Teacher Results- Practices Instrument (scale 1-4) Experimental Group (n = 36) Control Group (n = 49) Vocabulary Pre 2.81 2.76 Post 3.16 2.81 Reading Comprehension Pre 3.11 3.16 Post 3.36 3.21 Writing Pre 2.89 2.76 Post 3.31 2.89
- 34. 4 th Grade Teacher Results- Significance Subject ANCOVA Significance Effect Size Knowledge Total ELA Yes ( p <.01) Medium (.53) Vocabulary Yes ( p <.05) Small (.27) Reading Comprehension Yes ( p <.01) Medium (.55) Writing Yes ( p <.01) Medium (.75) Practice Vocabulary Yes ( p <.01) Small (.45) Reading Comprehension Yes ( p <.01) Small (.32) Writing Yes ( p <.01) Small (.43)
- 35. 4 th Grade Student Instruments Instrument Number of Items Pre-Test Reliability Post-Test Reliability Knowledge Total ELA 29 .829 .852 Vocabulary 10 .708 .738 Reading Comprehension 10 .659 .728 Writing 9 .59 .60 Practice Vocabulary 6 .287 .336 Reading Comprehension 10 .744 .760 Writing 10 .723 .746
- 36. 4 th Grade Student Results- Knowledge Instrument Experimental Group (n = 831) Control Group (n = 1225) ELA Pre 54% 53% Post 63% 60% Vocabulary Pre 64% 64% Post 73% 69% Reading Comprehension Pre 51% 50% Post 58% 54% Writing Pre 47% 46% Post 59% 56%
- 37. 4 th Grade Student Results- Practices Instrument Experimental Group Control Group Vocabulary Pre (scale 0-3) 1.84 n = 762 1.85 n = 919 Post (scale 0-3) 1.89 n = 750 1.89 n = 902 Reading Comprehension Pre (scale 0-3) 2.14 n = 765 2.11 n = 918 Post (scale 0-3) 2.20 n = 757 2.16 n = 919 Writing Pre (scale 1-3) 2.36 n = 757 2.33 n = 913 Post (scale 1-3) 2.44 n = 749 2.38 n = 908
- 38. 4 th Grade Student Results- Significance Subject ANCOVA Significance Effect Size HLM Significance Knowledge Total ELA Yes ( p <.01) None (.10) Yes Vocabulary Yes ( p <.01) None (.09) Yes Reading Comprehension Yes ( p <.01) None (.08) No Writing Yes ( p <.01) None (.09) No Practice Vocabulary No - No Reading Comprehension Yes ( p <.05) None (.06) No Writing Yes ( p <.05) None (.08) Yes
- 39. 5 TH GRADE RESULTS
- 40. 5 th Grade Teacher Instruments Instrument Number of Items Pre-Test Reliability Post-Test Reliability Knowledge Total Mathematics 31 .72 .80 Fractions 9 .54 .48 Algebraic Thinking 8 .55 .60 Measurement 14 .55 .67 Practice Fractions 4 .61 .60 Algebraic Thinking 8 .82 .82 Measurement 26 .82 .88
- 41. 5 th Grade Teacher Results- Knowledge Instrument Experimental Group (n = 34) Control Group (n = 45) Mathematics Pre 46% 45% Post 58% 44% Fractions Pre 60% 57% Post 63% 54% Algebraic Thinking Pre 47% 50% Post 61% 49% Measurement Pre 36% 34% Post 53% 34%
- 42. 5 th Grade Teacher Results- Practices Instrument (scale 1-4) Experimental Group (n = 34) Control Group (n = 45) Fractions Pre 2.74 2.87 Post 3.16 2.68 Algebraic Thinking Pre 2.87 3.01 Post 3.42 2.98 Measurement Pre 2.37 2.59 Post 2.91 2.74
- 43. 5 th Grade Teacher Results- Significance Subject ANCOVA Significance Effect Size Knowledge Total Mathematics Yes ( p <.01) Large (.71) Fractions Yes ( p <.05) Small (.29) Algebraic Thinking Yes ( p <.01) Medium (.43) Measurement Yes ( p <.01) Large (.93) Practice Fractions Yes ( p <.01) Large (.88) Algebraic Thinking Yes ( p <.01) Large (.75) Measurement Yes ( p <.01) Large (.57)
- 44. 5 th Grade Student Instruments Instrument Number of Items Pre-Test Reliability Post-Test Reliability Knowledge Total Mathematics 29 .73 .85 Fractions 8 .49 .68 Algebraic Thinking 10 .60 .73 Measurement 11 .48 .59
- 45. 5 th Grade Student Results- Knowledge Instrument Experimental Group (n = 648) Control Group (n = 790) Mathematics Pre 42% 44% Post 54% 53% Fractions Pre 30% 32% Post 48% 46% Algebraic Thinking Pre 43% 45% Post 54% 53% Measurement Pre 50% 51% Post 60% 59%
- 46. 5 th Grade Student Results- Significance Subject ANCOVA Significance Effect Size HLM Significance Knowledge Total Mathematics Yes ( p <.01) None (.10) No Fractions Yes ( p <.05) None (.09) No Algebraic Thinking Yes ( p <.05) None (.07) No Measurement No - No
- 47. 7 TH GRADE RESULTS
- 48. 7 th Grade Teacher Instruments Instrument Number of Items Pre-Test Reliability Post-Test Reliability Knowledge Total ELA 22 .731 .728 Vocabulary 12 .703 .700 Reading Comprehension 5 .475 .485 Writing 4 .120 .214 Practice Vocabulary 4 .473 .506 Reading Comprehension 13 .871 .886 Writing 17 .836 .833
- 49. 7 th Grade Teacher Results- Knowledge Instrument Experimental Group (n = 36) Control Group (n = 49) ELA Pre 34% 30% Post 40% 30% Vocabulary Pre 26% 21% Post 31% 22% Reading Comprehension Pre 37% 35% Post 45% 36% Writing Pre 47% 41% Post 54% 40%
- 50. 7 th Grade Teacher Results- Practices Instrument (scale 1-4) Experimental Group (n = 35) Control Group (n = 45) Vocabulary Pre 2.9 3.1 Post 3.2 3.2 Reading Comprehension Pre 2.6 2.6 Post 2.8 2.7 Writing Pre 3.0 3.1 Post 3.3 3.2
- 51. 7 th Grade Teacher Results- Significance Subject ANCOVA Significance Effect Size Knowledge Total ELA Yes ( p <.01) Medium (.48) Vocabulary Yes ( p <.05) Medium (.34) Reading Comprehension No - Writing Yes ( p <.01) Medium (.47) Practice Vocabulary Yes ( p <.05) Small (.30) Reading Comprehension No - Writing Yes ( p <.05) Small (.28)
- 52. 7 th Grade Student Instruments Instrument Number of Items Pre-Test Reliability Post-Test Reliability Knowledge Total ELA 24 .807 .833 Vocabulary 10 .701 .739 Reading Comprehension 10 .695 .720 Writing 4 .262 .438 Practice Reading Comprehension 12 .702 .773 Writing 21 .844 .863
- 53. 7 th Grade Student Results- Knowledge Instrument Experimental Group (n = 831) Control Group (n = 1225) ELA Pre 64% 66% Post 67% 66% Vocabulary Pre 70% 71% Post 72% 71% Reading Comprehension Pre 62% 63% Post 62% 61% Writing Pre 58% 59% Post 67% 66%
- 54. 7 th Grade Student Results- Practices * There was a significant difference between the pre-scores of the experimental and control groups. Instrument Experimental Group Control Group Reading Comprehension Pre (scale 0-1) .44 n = 789 .43 n = 1154 Post (scale 0-1) .50 n = 797 .46 n = 1162 Writing Pre (scale 1-3) 2.25 n = 830 2.22* n = 1224 Post (scale 1-3) 2.33 n = 826 2.24 n = 1225
- 55. 7 th Grade Student Results- Significance Subject ANCOVA Significance Effect Size HLM Significance Knowledge Total ELA Yes ( p <.01) None (.08) No Vocabulary Yes ( p <.05) None (.05) No Reading Comprehension Yes ( p <.05) None (.07) No Writing No - No Practice Reading Comprehension Yes ( p <.01) None (.11) Yes Writing Yes ( p <.01) None (.14) No
- 56. 8 TH GRADE RESULTS
- 57. 8 th Grade Teacher Instruments Instrument Number of Items Pre-Test Reliability Post-Test Reliability Knowledge Total Mathematics 46 .809 .847 Proportional Reasoning 29 .690 .772 Geometric Measurement 10 .430 .568 Functions 7 .597 .657 Practice Proportional Reasoning 16 .843 .883 Geometric Measurement 12 .772 .757 Functions 37 .947 .951
- 58. 8 th Grade Teacher Results- Knowledge Instrument Experimental Group (n = 28) Control Group (n = 43) Mathematics Pre 38% 34% Post 42% 35% Proportional Reasoning Pre 35% 33% Post 40% 34% Geometric Measurement Pre 41% 36% Post 47% 34% Functions Pre 47% 37% Post 49% 40%
- 59. 8 th Grade Teacher Results- Practices Instrument (scale 1-4) Experimental Group (n = 28) Control Group (n = 43) Proportional Reasoning Pre 2.7 2.7 Post 2.9 2.6 Geometric Measurement Pre 2.6 3.8 Post 2.9 3.8 Functions Pre 2.5 4.0 Post 2.9 3.9
- 60. 8 th Grade Teacher Results- Significance Subject ANCOVA Significance Effect Size Knowledge Total Mathematics Yes ( p <.01) Small (.29) Proportional Reasoning Yes ( p <.05) Small (.28) Geometric Measurement Yes ( p <.01) Medium (.43) Functions No - Practice Proportional Reasoning Yes ( p <.01) Large (.54) Geometric Measurement Yes ( p <.05) Medium (.34) Functions Yes (p<.01) Large (.51)
- 61. 8 th Grade Student Instruments Instrument Number of Items Pre-Test Reliability Post-Test Reliability Knowledge Total Mathematics 36 .704 .758 Proportional Reasoning 10 .416 .440 Geometric Measurement 10 .493 .642 Functions 16 .499 .531
- 62. 8 th Grade Student Results- Knowledge Instrument Experimental Group (n = 799) Control Group (n = 1090) Mathematics Pre 50% 47% Post 52% 48% Proportional Reasoning Pre 59% 56% Post 55% 53% Geometric Measurement Pre 43% 40% Post 47% 43% Functions Pre 43% 40% Post 52% 48%
- 63. 8 th Grade Student Results- Significance Subject ANCOVA Significance Effect Size HLM Significance Knowledge Total Mathematics Yes ( p <.01) None (.01) Yes Proportional Reasoning No - No Geometric Measurement Yes ( p <.01) None (.07) Yes Functions Yes ( p <.01) None (.13) Yes
Notes de l'éditeur
- Little background
- Examining mean scores for Teachers, there were generally little difference in scores on the pre-measures. Larger differences were apparent for the post-measures, with the Experimental group always performing higher than the control group.
- Several steps were taken while performing analyses. These steps began by examining the reliability of each teacher and student scale score. Reliability is measured on a scale from 0 to 1, with 0 being least reliable and 1 being completely reliable. Values of reliability estimates >.70 are considered optimal, however reliability may decrease as the number of items used to form a scale decreases. Next, we examined mean scores for each group (experimental and control group) within each grade level. When calculating mean scores for all knowledge tests, we focused on the percent of the total possible points a participant received (e.g., percent correct). For test scores, a higher percent correct represents higher performance. When calculating mean practice scores, we calculated the mean scale score. For all practice items, the scale ranged from 1 to 4, with 1 representing that a practice never occurs and 4 representing very frequent use of the practice. For the practice scores, a higher score represents more frequent use of a set of instructional practices which the course developers deemed to be desirable practices.
- Teachers were the primary targets of the OPD. To examine teacher outcomes, two sets of analyses were employed: 1. Analysis of covariance allowed us to examine changes in post-scores while taking into account any difference in pre-scores. These analyses provided insight into the statistical significance of post-test score differences. 2. Effect sizes allowed us to examine the practical significance of any differences in post-test scores. It is possible for a score difference to have statistical significance, but for the difference to have little practical significance.
- Students were the secondary target of the OPD. That is, by changing teacher knowledge and practices, the OPD was intended to have a secondary effect on student achievement in the targeted areas. It is important to note, however, that relatively little time was provided for any changes in teachers to translate into effects on students, so we anticipate that student effects will be less frequent and smaller than teacher effects. To examine student effects, three methods were employed: 1. Analysis of covariance, which examined differences between groups in post-test scores adjusting for any differences in pre-test scores 2. Effect sizes and [go to next slide]
- 3. Hierarchial linear modeling. his technique allowed us to adjust score differences to account for any effects that occurred within classrooms as opposed to between classrooms. This technique is appropriate when students are nested or clustered within teachers. .
- Reliability. With the exception of the Writing Knowledge test, the instruments were generally reliable.
- Examining mean scores for Teachers, there were generally little difference in scores on the pre-measures. Larger differences were apparent for the post-measures, with the Experimental group always performing higher than the control group.
- A similar pattern is seen for the Teacher Practices. There were relatively small differences on the pre-measures, and larger differences on the post-measures, with the Experimental group consistently reporting more frequent use of targeted practices.
- Examining differences in mean post-scores, adjusting for any differences in pre-scores, we see that the Experimental group scored statistically significantly higher on all post-measures, and the effect sizes ranged from .27 to .75. An effect size of .27 indicates that the Experimental group performed approximate one quarter of a standard deviation higher than the control group.
- Examining the reliability of the Student Instruments, we see that all instruments, except the Vocabulary Practices scale, had adequate reliabilities.
- Focusing on the Knowledge scales, we see little differences between the Experimental and Control groups on the pre-measures and small differences on the post, with differences on the post consistently favoring the Experimental group. These differences, however, are relatively small.
- For the Student Practices measures, we see little differences on the pre-measures and most post-measures. The largest difference occurs for the Writing Post measures, with the experimental group reporting more frequent use of targeted practices.
- After adjusting for differences in pre-scores, we see that most score differences are statistically significant and favor the experimental group. In all cases, however, the size of the differences is relatively small. In addition, when clustering of students within teachers is taken into account, the Total Knowledge, Vocabulary Knowledge, and the Writing Practices mean score differences remain statistically significant. Collectively, these findings suggest that the OPD intervention generally had a positive effect on teacher knowledge and practices and that these effects translated to effects on students in some, but not all, areas of knowledge and practice.
- Reliability of the 5th grade Teacher instruments was generally lower than the 4th grade.
- Like the 4th grade, there were generally little differences on the Teacher pre-test measures, but noticeable difference on the post-measures. In all cases, Teachers in the Experimental group performed higher on the post-measure than teachers in the Control group.
- A similar pattern was seen for the targeted practices. In general, there were small differences in the pre-measures and larger differences in the post-measures, with post-measure differences favoring the teachers in the Experimental group.
- The Analysis of Covariance indicated that all post-measure differences were statistically significant. In addition, the effect sizes were generally sizable, ranging from .29 to .93.
- The reliability of the 5th grade student measures was also lower than that of the 4th grade instruments. Reliability of the post-measures was generally higher than the pre-measures, and was highest for the Total test score.
- For the pre-measures, the Control group scored slightly higher on all measures. For the post-measures, this pattern reversed such that the Experimental group performed slightly higher on all measures.
- When examining statistical significance, the Analysis of Covariance results indicate that the small differences in post-scores were statistically significant for all but the Measurement scale. These differences, however, resulted in small effect sizes and were not significant once clustering of students within teachers was taken into account. Overall, the 5th grade findings suggest that the OPD courses had positive effects on Teachers’ knowledge and practices, and that these effects translated into smaller, and less meaningful, effects on students knowledge than was seen in the 4th grade ELA study.
- The reliability of the 7th grade instruments varied widely with the Knowledge Total ELA and Vocabulary and the Reading Comprehension and Writing practices having adequate levels of reliability, while the other instruments had poor reliability. It is important to note that those scales that had low reliability also contained a relatively small number of items.
- Comparison of Teacher Knowledge scores indicate that the the Experimental group consistently performed slightly higher on the pre-measures than the Control group. These differences, however, were not statistically significant. The Experimental group also performed noticeably higher on the post-measures.
- For the Teacher Practices, there were not consistent differences between the two groups on either the pre or post measures.
- The Analysis of Covariances indicate that, after adjusting for differences on the pre-measures, the experimental group mean scores were statistically significantly higher on the majority of measures. The effect sizes for these measures ranged from .28 to .48.
- For the student measures, the reliability was adequate for all scales except writing.
- Focusing on mean scores, the Experimental group consistently performed slightly lower on the pre-measures than did the control group. These differences, however, were not statistically significant. On the post measures, however, the pattern reversed such that the Experimental group consistently performed slightly higher. All observable differences for the pre and post measures were relatively small.
- For the student practices, the experimental group consistently reported more frequent practices for both the pre and post measures. The magnitude of the differences was larger for the post-measures than for the pre-measures.
- The Analysis of Covariances indicate that after adjusting for differences in pre-scores, all post-score differences, except writing, were statistically significant. The effect sizes, however, were small and when clustering of students within teachers is taken into account, only difference in Reading Comprehension practices remains statistically significant. Collectively, the 7th grade findings suggest that the OPD courses has a positive effect on Teachers’ knowledge and practices in the targeted areas and that these effects generally translated to small effects on students practices and knowledge in the targetted areas.
- With the exception of one instrument, the Teacher measures had adequate levels of reliability.
- Examining differences in mean scores, the Experimental group scored higher on both the pre and post measures. Many of the differences were larger for the post-measures than they were for the pre.
- For the targeted practices, the Experimental group generally reported the same or lower frequency of use of targeted practices for the pre-measures. For proportional reasoning, experimental teachers reported more frequent use of the targeted practices for the post measure. For the remaining practices, the magnitude of differences decreased such that the experimental group became more similar to the control group.
- When adjusting post-scores for differences in pre-scores, all but one measure indicated that the Experimental group had larger mean score gains than did the control group. The effect sizes ranged from .28 to .54.
- With the exception of the Total Mathematics score, the student measures had low reliability.
- Examining differences in mean scores, the Experimental group consistently performed about 3 percentage points higher than the control group on each pre-measure. The score differences increased slightly for the post-measures.
- Adjusting for differences in pre-scores, the Analysis of Covariances indicate that there were significant mean score differences for three of the four measures. While the effect sizes were generally small, the score difference remained statistically significant after the clustering of students within teachers was taken into account. Collectively, the 8th grade findings suggest that the OPD courses generally had positive effects on the targeted practices and knowledge of teachers and that these changes in teachers translated into small, but statistically significant, effects on student knowledge.