2. I can’t explain it.
It just wasn’t an A
paper.
~ pre-rubric educators
3. Getting the Lay of the Land:
Defining The Jargon
Rubric
a guide used to score performance assessments
in a reliable, fair, and valid manner
generally composed of:
dimensions for judging student performance
a scale for rating performances on each
dimension
standards of excellence for specified
performance levels
(SRI International)
4. Why Rubrics?
¡ Provide students with expectations about what
you will assess
¡ Inform students on the standards they must
meet/work towards meeting
¡ Indicate to students where they are in relation to
course/program goals
¡ Increase your consistency in ratings or
performance, products, or understanding
¡ Gather data to support grades
5. Jargon Cont’d
Authentic Assessment
meaningful, real-life learning experiences
includes:
recording evidence of the learning process
applications in products and performances
integrations of new knowledge
reflecting on one's own progress
interpreting meaning
(Herberger College of the Arts, Arizona State University)
6. Jargon Cont’d
Analytic Rubric: outline or list of major elements
that students should include in a finished work
Highly prescriptive
Holistic Rubric: less objective than analytic; levels
pre-determined and you assign
Highly subjective
Annotated Holistic Rubric: hybrid of above;
defined quality levels plus commentary
Reduces ambiguity, increases efficiency, and
allows students to see road to improvement
(IMHO)
7. Validity is Key
Reliability: measures educational objectives
as consistently as possible
Relevance: measures educational objectives
as directly as possible
Utility: provides formative or summative
results effectively - clear implications for
evaluation and improvement
8. If the only tool you
have is a hammer, you
tend to see every
problem as a nail.
~ Abraham Maslow
9. Add Rubrics to Your Toolkit;
Don’t Throw Out Other Tools
Rubrics are best used when:
Assignments are multi-faceted; combining
lower and higher order skills
Your subjectivity is/could be called into
question
Assessing an action or combination of actions
rather than a thing
10. Let’s Not Reinvent the Wheel
There are current and authoritative resources that can
save you immense amounts of time
Ontario College Writing Exemplars
developed by the Heads of Language (HOL) with
funding from School/College/Work Initiative
program of the Ontario Ministry of Education
College Diploma and Certificate Program Standards
from the Ministry of Training, Colleges, and
Universities
11. 2. Showing You the Sights
We’re taking the economy
tour…
Five Questions – That’s It!
12. Question 1: What dimensions ensure
highest quality?
Hint: Can include knowledge, skills & abilities/
Content specific or life-long goals
Consideration: Students may experience
difficulty with course specific mixing with life-
long goals
Most Common Misstep: Learning outcomes
don’t match assessment
LO = critical thinking; assessment
dimensions = format, mechanics, and
citation style
13. Some Usual Dimensions
From high school, students are familiar with
categories:
Knowledge and Understanding
Thinking
Communication
Application
14. Some Usual Dimensions
Or adopt a learning theory
Bloom’s Taxonomy (or Big 6
Revised Bloom’s)
Knowledge Task Definition
Comprehension Info Seeking Strategies
Application Location and Access
Analysis Use of information
Synthesis Synthesis
Evaluation Evaluation
15. Question 2 How many levels of
achievement/performance to include?
Hint: Give yourself some wiggle room
Consideration: Letters vs. levels vs. descriptors
A, B, C, D vs. 1, 2, 3, 4 vs. unacceptable,
marginal, proficient, exemplary vs. novice,
apprentice, proficient, distinguished
Most Common Misstep: Using too many levels
of achievement
16. Question 3: What is a clear
description at each level?
Hint: Try to determine qualitative differences
that characterize work or performance. Start
with B/acceptable/proficient level
Consideration: Comparative language alone
fails to highlight unique features, but using
unique language may connote different
meanings
Most Common Misstep: Including value
laden terms that showcase judgement, but
little guidance
17. Question 4: What rating scheme/
weighting of dimensions do I use?
Hint: Add this in a way that fits with your
philosophy and course requirements
Consideration: Different assignments may
measure the same dimensions in differing
degrees. One rubric could serve an entire
course.
Most Common Misstep: Using a weighted
rating in your head, but not communicating it
to the students
18. Question 5: What worked and
what didn’t?
Hint: Do a trial run with colleague(s) rather than
one, entire class
Consideration: Do you need more focus on
content, format, delivery? Was one
dimension weighted too heavily? Etc…
Most Common Misstep: Viewing rubric as a
permanent panacea
19. Rubrics Recap
Decide which assignments suit a rubric
Use our 5 questions as a checklist or
frame
Get help/feedback/constructive criticism
whenever and wherever you can
From colleagues
From students
From the literature
20.
21. Research Paper Grading Rubric
For Research Component
Uses:
6.To set performance expectations by distributing to students when a paper is assigned.
7.To evaluate the portion of a student’s paper related to research and information use.
Beginning Proficient Advanced
Score
(0-12 points) (13-16 points) (17-20 points)
Formulates a question that is
Formulates a research question that is
1. Determine the Creates an unfocused or focused and clear. Student
focused, clear, and complete and identifies
Extent of the unmanageable research question. identifies concepts related to the
key concepts. Student identifies most or all
Information Needed Student identifies few or no topic, and identifies some useful
relevant information tools in various
(20 points) relevant information tools. information tools to meet the
potential formats.
information need.
Executes an appropriate
2. Access the Needed research strategy. Student solves Implements a clear and focused research
Uses information tools poorly and
Information problems by finding a variety of strategy, uses tools effectively, and finds
gathers information that lacks
Effectively relevant information resources information that directly fulfills the
relevance, quality, and balance.
(20 points) and evaluates search information need.
effectiveness.
Examines information using
criteria such as authority,
3. Evaluate Uses inadequate criteria to judge Compares and evaluates multiple and
credibility, relevance, timeliness,
Information and Its information quality. Student makes diverse sources and viewpoints according to
and accuracy, and makes good
Sources Critically little effort to examine the specific criteria appropriate for the
judgements about
(20 points) information located for reliability. discipline.
what to keep and what to
discard.
4. Use Information Shows little evidence of Often uses appropriate Effectively synthesizes and integrates
Effectively to incorporating information into information and evidence to information from a variety of sources, draws
Accomplish a their knowledge base. Student uses support their claims and appropriate conclusions, and clearly
Specific Purpose information poorly to accomplish a conclusions and to accomplish a communicates ideas to others to accomplish
(20 points) specific purpose. specific purpose. a specific purpose.
5. Use Information
Inadequately cites ideas and Cites ideas and information of Consistently and accurately cites ideas and
Ethically
information of others. others with few errors. information of others.
(20 points)
22. Analytic Rubric Example
LEVEL OF ACHIEVEMENT/PERFORMANCE
CRITERIA/
DIMENSION Excellent Good Needs Improvement
Load supported greater than 12 kg 6 to 12 kg less than 6 kg
Weight of bridge less than 30 grams 30 to 60 grams over 60 grams
Bridge span greater than 385 mm 360 to 385 mm less than 360 mm
stops 3 or more times
Ability of matchbox car
does not stop stops 1 or 2 times or cannot roll entire
to roll across bridge
length of bridge