This session describes how a selective, public STEM high school has used the College and Work Readiness Assessment (CWRA) in efforts to model student performance and demonstrate achievement gains in critical thinking, analytical reasoning, problem solving, and written communication. In the context of both diagnostic utility and repeated measures student growth, we will describe results from over 400 students and explain how this new measure of high order thinking skills provides reliable results for advanced students. We will also explore the institutional value of having normed data for value-added measures.

1. MEASUREMENT AND
MODELING OF HIGHER
ORDER SKILLS IN STEM
TALENT DEVELOPMENT
USING THE CWRA
Christopher Kolar
NCSSSMST/NAGC Annual Convention 2013, Indianapolis

2. The challenge



Exceptional
students
Exceptional claims
of exceptional
results
Exceptional
Evidence

3. Assessment at IMSA
Standards of Significant Learning (SSLs)

4. Competence In Inquiry



Making accurate
observations.
Evaluating
evidence.
Challenging
assumptions.



Making connections.
Making sound
judgments.
Communicating
powerfully.

5. CWRA: Performance task and document

6. In addition to
an overall
Performance
Task score,
subscores
include:
Analytic
Reasoning
Writing
Effectiveness
Writing
Mechanics
Problem
Solving

8. MEASURING UP
Admissions data

9. Admissions Benchmarking





Prior GPA
SAT-M
SAT-CR
SAT-We
SAT-Wmc

10. Admissions Benchmarking
CWRA
performance
task score
significantly
correlated
with all
admissions
metrics.





Prior GPA
SAT-M
SAT-CR
SAT-We
SAT-Wmc

11. MEASURING UP
Initial academic performance

12. First Year Grade Point Average
First year
grades in
writing
intensive
courses:
Literary
Explorations,
American
Studies, and
Scientific
Inquiries
(Biology)





Prior GPA
SAT-CR
SAT-We
SAT-Wmc
CWRA+

13. First Year Grade Point Average
First year
grades in
writing
intensive
courses:
Literary
Explorations,
American
Studies, and
Scientific
Inquiries
(Biology):
23% of
variance
explained.





Prior GPA
SAT-CR
SAT-We
SAT-Wmc
CWRA+

14. MEASURING UP
English core assessment

15. No reliability as outcome measures
IMSA



Thesis
Paragraphs
Quote Usage
CWRA+
Analytic
Reasoning
 Writing
Effectiveness
 Writing Mechanics
 Problem Solving


16. THEREFORE
What this means in practice

17. Student Support
 CWRA+
individual results as a diagnostic
 Further grade and retention modeling using this
data

18. Assessment Development



Performance task academy
School improvement plan
Revision of core assessment

19. Value added


“Small” effect size for time by junior year.
Senior exit testing next.