Using DesCartes Instructional Ladders to Plan for Differentiated Instruction
Using tests for teacher evaluation texas
1. Using tests for high stakes evaluation, what
administrators need to know
TASA Midwinter Conference –
January 29, 2013
Austin, Texas
John Cronin, Ph.D.
Director
The Kingsbury Center @ NWEA
2. Using tests for high stakes evaluation,
what educators need to know.
Presenter - John Cronin, Ph.D.
Contacting us:
Rebecca Moore: 503-548-5129
E-mail: rebecca.moore@nwea.org
This PowerPoint presentation and recommended resources are
available at our SlideShare website:
http://www.slideshare.net/NWEA/tag/kingsbury-center
3. Much of the nation has moved from a
model of education reform that
focused on fixing schools to a model
that is focused on fixing the teaching
profession. Texas has not yet joined
this bandwagon, but administrators
need to understand the issues to
contribute to the public dialogue.
4. NWEA’s position on the use of
tests for teacher evaluation
• The principal or designated evaluator should control the
evaluation of teachers.
• Multiple sources of data should inform this evaluation
including:
– Classroom observation
– Evidence of student achievement
– Professional participation
• Tests may inform the evaluation process but should be a
controlling factor
5. Does NWEA participate in the use
of tests for teacher evaluation?
• We serve 5200 school systems throughout the United States.
• In many states, the use of local assessments is required as
part of the teacher evaluation.
• In these states we:
– Advise our partners on the issues associated with this issue.
– Work with partners to implement common solutions that are in
compliance with state mandates, fair to all stakeholders, and legally
defensible.
– Form state advisory groups of school systems to provide input to this
process, implement a common solution, and establish a network for
schools to get support from colleagues.
6. Primary sources of teacher
dismissal
• Top source – Economic layoff
• Second source – non-renewal of probationary
teachers
• Third source – dismissal for misconduct
• Fourth source – dismissal for incompetence.
7. What are the primary issues in using
tests for teacher evaluation
• Alignment between tests and curriculum
expectations
8. Issues in the use of tests for teacher
evaluation – curriculum alignment
1. Assessment of high school
subjects
2. Assessment of subjects without
assessments
9. Issues in the use of tests for teacher
evaluation – curriculum alignment
Assessment of subjects without
assessments
Music, art, PE, and many other
courses lack appropriate
assessments to measure
student learning and gains.
10. Issues in the use of tests for teacher
evaluation – curriculum alignment
Common Solutions that are
problematic
Solution - Hold all teachers
accountable for basic skills
Problem – This distracts
specialized teachers from their
core responsibilities and is not a
fair assessment of their job
performance
11. Issues in the use of tests for teacher
evaluation – curriculum alignment
Common Solutions that are
problematic
Solution – Evaluate by having
teachers develop SLO’s or
Student Learning Objectives
Problem – This creates a huge
evaluation load on
administrators and SLO’s are
often not rigorous or targeted to
all students
12. Issues in the use of tests for teacher
evaluation – curriculum alignment
Assessment of high school
subjects
We are aware of one district in the
United States that assesses all
courses. They manage 2600
tests.
State and standardized math and
reading tests do not always
closely align to the expectations
of math and English courses
13. How the teacher’s contribution to learning is
commonly measured by tests
Is the progress produced by this
teacher dramatically greater or
less than teaching peers that
deliver instruction to comparable
students?
14. Moving from Proficiency to Growth
All students count when
accountability is measured
through growth.
15. One district’s change in 5th grade math performance
relative to Kentucky cut scores
proficiency college readiness
16. Number of 5th grade students meeting math growth
target in the same district
17. Issues in the use of growth and value-
added measures
Measurement design of the
instrument
Many assessments are not
designed to measure growth.
Others do not measure growth
equally well for all students.
18. Tests are not equally accurate for all
students
California STAR NWEA MAP
19. Tests are not equally accurate for all
students
Grade 6 New York Mathematics
21. Issues in the use of growth and value-
added measures
“Among those who ranked in the top
category on the TAKS reading test, more
than 17% ranked among the lowest two
categories on the Stanford. Similarly
more than 15% of the lowest value-added
teachers on the TAKS were in the highest
two categories on the Stanford.”
Corcoran, S., Jennings, J., & Beveridge, A., Teacher Effectiveness on High and Low Stakes
Tests, Paper presented at the Institute for Research on Poverty summer workshop, Madison, WI
(2010).
22. Issues in the use of growth and value-
added measures
Instability of results
A variety of factors can cause value-
added results to lack stability.
Results are more likely to be stable
at the extremes. The use of
multiple-years of data is highly
recommended.
23. Issues in the use of growth and value-
added measures
Instability of results
Los Angeles Times Study
24. Reliability of teacher value-added
estimates
Teachers with growth scores in lowest and
highest quintile over two years using NWEA’s
Measures of Academic Progress
Bottom Top quintile
quintile Y1&Y2
Y1&Y2
Number 59/493 63/493
Percent 12% 13%
r .64 r2 .41
Typical r values for measures of teaching effectiveness range
between .30 and .60 (Brown Center on Education Policy, 2010)
25. Issues in the use of growth and value-
added measures
Control for statistical error
All models attempt to address this
issue. Nevertheless, many teachers
value-added scores will fall within
the range of statistical error.
26. Issues in the use of growth and value-
added measures
Control for statistical error
New York City
28. Issues in the use of growth and value-
added measures
Lack of random assignment
The use of a value-added model
assumes that the school doesn’t
add a source of variation that isn’t
controlled for in the model.
e.g. Young teachers are assigned
disproportionate numbers of
students with poor discipline
records.
29. Potential Litigation Issues
The use of value-added data for high stakes
personnel decisions does not yet have a strong,
coherent, body of case law.
Expect litigation if value-added results are the
lynchpin evidence for a teacher-dismissal case
until a body of case law is established.
30. Possible legal issues
• Title VII of the Civil Rights Act of 1964 –
Disparate impact of sanctions on a protected
group.
• State statutes that provide tenure and other
related protections to teachers.
• Challenges to a finding of “incompetence”
stemming from the growth or value-added
data.
31. Other issues
Security and Cheating
When measuring growth, one
teacher who cheats disadvantages
the next teacher.
32. Cheating
Atlanta Public Schools
Crescendo Charter Schools
Philadelphia Public Schools
Washington DC Public Schools
Houston Independent School
District
Michigan Public Schools
36. Security considerations
• Teachers should not be allowed to view the contents
of the item bank or record items.
• Districts should have policies for accomodation that
are based on student IEPs.
• Districts should consider having both the teacher and
a proctor in the test room.
• Districts should consider whether other security
measures are needed for both the protection of the
teacher and administrators.
37. Recommendations
• Embrace the formative advantages of growth
measurement as well as the summative.
• Create comprehensive evaluation systems with
multiple measures of teacher effectiveness (Rand,
2010)
• Select measures as carefully as value-added models.
• Use multiple years of student achievement data.
• Understand the issues and the tradeoffs.
38. Thank you for attending
Presenter - John Cronin, Ph.D.
Contacting us:
NWEA Main Number: 503-624-1951
E-mail: rebecca.moore@nwea.org
The presentation and recommended resources are
available at our SlideShare site:
http://www.slideshare.net/NWEA/tag/kingsbury-center
Notes de l'éditeur
Race to the Top, Gates Foundation, Teach for America…
Beaverton Oregon laid off 204 of their 2300 teachers this fall. This dwarfs the number of teachers who have been laid off for incompetence over more than a decade.
Beaverton Oregon laid off 204 of their 2300 teachers this fall. This dwarfs the number of teachers who have been laid off for incompetence over more than a decade.
Beaverton Oregon laid off 204 of their 2300 teachers this fall. This dwarfs the number of teachers who have been laid off for incompetence over more than a decade.
