3. 21st CTL Evaluation
Since March 2007, the 21st CTL
Evaluation Project has collected:
• 5,500 21st CTL Classroom Observations
• 55,000 Teacher Surveys
• 650,000 Student Surveys
• 18,000 PATI Surveys (Instructional
Phase)
4. Analysis
Start
of
Program
Spring
2010
(Fall
2007/2008)
• The same 21st CTL teachers compared at
different points in time.
• Data collection periods (fall/spring 2007 -
2010)
• Cohorts (teachers with different number of
years in program)
5. Instruments
Observations
Teacher/Student Surveys
PATI (Instructional Phase)
http://cff.psu.edu
7. Has 21st CTL Changed...
• complexity of class content
• relevance of class content
• instructional style
8. Overall
Complexity of Basic
Higher Order
Content Skills
Relevance of
Artificial Real World
Content
Instructional
Didactic Constructivist
Style
9. Change in the
Complexity of Content
21st CTL teachers now focus more
on higher order topics than they did
at the beginning of the program.
10. Complexity of Content
Describe the content your class is designed to convey.
(Almost All Higher Order Skills / More Higher Order Skills
than Content Knowledge Combined)
22%
Teacher
19%
Survey
Cohort 1
16% Cohort 2
Cohort 3
Differences
13% 5.89%
6.85%
4.65%
10%
Start of Program Spring 2010
11. Complexity of Content
Percent of Time Spent in Really Complex Thinking / Problem Solving
(Quite a Lot / Almost All the Time)
43%
Student
41%
Survey
Cohort 1
39% Cohort 2
Cohort 3
Differences
37%
2.52%
2.00%
1.74%
35%
Start of Program Spring 2010
12. Complexity of Content
Overall Comparison by Class Period - Basic - Higher Order
(Ratings of 5, 6, and 7 Combined)
70%
66% Observations
62% First Third
Middle Third
Last Third
58%
Differences
9.12%
54%
8.35%
4.06%
50%
Start of Program Spring 2010
13. Classroom Content is
More Authentic
21st CTL teachers report students
use technology to solve real world
problems more often than non-21st
CTL teachers.
14. Relevance of Content
Artificial - Real World
How would you describe the work that students do?
Cohort 1 Cohort 2 Cohort 3
4 Years 3 Years 2 Years
Fall 2007 Spring 10 Fall 2007 Spring 10 Fall 2008 Spring 10
Exercises &
17.48% 12.45% 19.71% 12.36% 16.67% 12.04%
Assignments
Blend First / Last 42.48% 35.65% 45.98% 39.61% 47.97% 42.95%
Even Balance 30.22% 35.29% 25.59% 35.76% 28.73% 34.61%
Blend First / Last 7.40% 13.31% 7.40% 10.04% 6.10% 8.21%
Projects and 2.43% 3.30% 1.32% 2.23% 0.54% 1.65%
Products
Fewer Exercises Fewer Exercises Fewer Exercises
More Even Balance More Even Balance More Even Balance
More Projects More Projects More Projects
Teacher Survey
15. Relevance of Content
Artificial - Real World
How often do students use technology to solve real-world problems?
Daily
PATI
Survey
(Instructional
Weekly Phase)
Cohort 1 - 4 Years in 21st CTL
Cohort 2 - 3 Years in 21st CTL
Cohort 3 - 2 Years in 21st CTL
Non-21st CTL
Monthly
0% 5% 10% 15% 20% 25% 30%
16. Relevance of Content
Artificial - Real World
Does the Lesson have a “Real World” Context (authenticity)?
(More real world, rating of 5, 6, or 7 combined)
Cohort 3 (2 Years in 21st CTL)
Start of Program 32.52%
First Third of the
Spring 2010 65.18%
Class
Difference 32.67%
Start of Program 36.45%
Middle Third of the
Spring 2010 70.15%
Class
Difference 33.70%
Start of Program 35.47%
Final Third of the
Spring 2010 70.16%
Class
Difference 34.68%
Artificial 1 2 3 4 5 6 7 Real World
Observation
17. Instructional Style is
Changing
21st CTL has changed the way many
teachers teach; increasing
percentages of teachers identify
their style as constructivist.
18. Instructional Style
Didactic - Constructivist
How would you describe your teaching style?
Cohort 1 Cohort 2 Cohort 3
4 Years 3 Years 2 Years
Fall 2007 Spring 10 Fall 2007 Spring 10 Fall 2008 Spring 10
Didactic 29.63% 26.34% 33.15% 26.21% 34.01% 29.68%
Even Balance 47.64% 52.93% 48.53% 51.83% 50.41% 52.39%
Constructivist 22.73% 20.73% 18.32% 21.96% 15.58% 17.92%
Less Didactic Less Didactic Less Didactic
More Even Balance More Even Balance More Even Balance
Less Constructivist More Constructivist More Constructivist
Teacher Survey
19. Instructional Style
Didactic - Constructivist
Class Period Term Mean Difference
Fall 09 3.951
First Third
Spring 10 4.365 0.414
Fall 09 4.372
Middle Third
Spring 10 4.801 0.429
Fall 09 4.489
Last Third
Spring 10 4.854 0.365
Didactic 1 2 3 4 5 6 7 Constructivist
Observation (Overall)
20. Instructional Style
Didactic - Constructivist
Who makes decisions about... (Teacher Completely / Teacher Mostly)
100%
75%
Student
Survey
50% Spring 2010
How Assignments Graded
Topics Studied
25% Topics of Papers or Assignments
Way Topics Studied
Working Together or Alone
Cohort 1 0%
Cohort 2
Cohort 3
23. Instructional Strategies
Teaching Strategies (Valuable / Very valuable)
97%
95%
92%
Teacher
90%
Survey
87%
Spring 2010
85%
Cohort 1 Cohort 2 Cohort 3
Teacher Led Discussion - High Level Outcomes Problem Based Learning
Authentic Learning Teacher Lecture
24. Use of Strategies
Rate the Level of Your Current Use of the Strategy (High)
50%
40%
30% PATI
Survey
20% (Instructional
Phase)
10%
Spring 2010
0%
Cohort 1 Cohort 2 Cohort 3 Non-21st CTL
Collaborative Learning Differentiation of Learning
Interactive Instruction Independent Study
25. Use of Strategies
Rate the Level of Your Current Use of the Strategy (High)
60%
45%
PATI
30% Survey
(Instructional
Phase)
15%
Spring 2010
0%
Cohort 1 Cohort 2 Cohort 3 Non-21st CTL
Direct Instruction Mediating Student Thinking
Inquiry Experiential Learning
26. ‣ What has been the impact of 21st CTL / CFF on
teaching in your school?
27. ‣ What impact do these changing teaching practices
have on student outcomes?
29. Thank you.
Image Attribution:
School Bell Tower
http://www.flickr.com/photos/road_less_trvled/2061666068/
by road_less_trvled
Taken October 13, 2007 in Zionsville, PA
http://creativecommons.org/licenses/by-nc/2.0/deed.en
Classroom with Desks against Wall
http://www.flickr.com/photos/ransford/770039665/
by Ben Ransford
Taken on June 9, 2007
http://creativecommons.org/licenses/by-nc/2.0/deed.en
