Presented by Aimee F. Strang, Pharm.D., MS-HPEd, Assistant Dean of Curricular Assessment, Albany College of Pharmacy and Health Sciences
Rubrics are powerful assessment tools. Well written rubrics guide student learning and provide formative feedback for improvement. However, creating valid rubrics can be a challenge. This presentation will focus on how to create rubrics within the ExamSoft suite of software, and tips for making them as useful as possible. Topics include selecting appropriate criteria, creating differentiating descriptions, and assigning grading schemas. Emphasis will be placed on ensuring rubrics measure student learning as defined by the course objectives.
2. Objectives
Recognize common flaws
Describe characteristics of good criteria
Write descriptions that distinguish levels of
performance
Use ExamSoft to assign grades
Ensure rubrics evaluate learning
2
6. 1. Focus on the Learning, Not the Activity
Start with the learning objectives
Create an assignment that demonstrates performance
of understanding
Be wary of engaging busy work
LO: Analyze current trends that affect faculty and
institutions in academic health professions
Create a jeopardy game
Create a poster presentation
6
7. 2. Select Good Performance Criteria
Demonstrate evidence of learning or meeting objective (not
achievement of assignment tasks)
Criteria can be named, defined, described and observed
Vary along a continuum of quality
Unidimensional or distinct
LO: Triage a patient to the appropriate level of care
History
Medication History
Triage
Communication
7
8. 3. Differentiate Levels of Performance
Start with a description of minimum proficiency
Describe a continuum
Yes Yes but No but No
How many levels?
LO: Recommend most appropriate cold medication to a
patient
Medication choice
8
9. 4. Quality Check
Grade past assignment
Get peer feedback
Get student feedback
Make improvements each time
9
11. Don’t Measure More Than 1 Criteria at a Time
Originality
• Develops a project that
is unique specifically
targeting a select
population
• Project presentation
delivery is designed for
a select population
• Develops a
project that
targets a select
population
• Project
presentation
delivery is
generic
• Develops a
generic project
• Project does not
target a select
population
Professional
Language
• Appropriate use of
professional language
and lay terms identified
as necessary for the
targeted population
• Spelling and grammar
are correct
• The balance
between
professional
language and lay
terms is
inappropriate
• Minimal spelling
and grammar
mistakes
• Unprofessional
language is used
often (texting
language)
• Many spelling
and grammar
mistakes
11
12. Don’t Create Criterion for Nonessential
Features
Neatness
Design of cover page
Audience engagement
Format of references
12
13. Don’t Score by Counting
13
Critical
Elements
Missing four or more of the
following:
• Background & description
• Market analysis and strategy
• Operational structure and
process
• Financial projections and
feasibility
• Action plan for implementation
• Critical risks and opportunities
• Conclusion/summary
Missing less
than 4
Includes all of
the following
14. Don’t Score Things Not Part of the
Rubric
Professionalism Acted in a
professional
manner (eye
contact,
empathy,
open posture)
Acted
professional
but needs
some
improvement
Acted
professionally
but need
significant
improvement
Did not act
professionally
14
15. Don’t Score the Product
15
Assignment Rubric Criterion
Reflect on today’s
learning
Stages of reflection
Write paper on history of
pharmacy
Organization, spelling,
length
Counsel patient on use of
eye drops
Introduction, body
language, eye contact
16. Don’t Codify Assignment Directions
into Rubric
Performance
Level
Needs
Improvement
Satisfactory Excellent
Contribution
and
Participation
Not apparent that
all the students in
the group
contributed. One
or two people
appear to have
done all the work.
Student dissent
on levels of
contribution.
Apparent that
some of the
students
contributed a fair
share to the
project, though
some workloads
varied. Students
made an effort to
include all group
members in the
process.
Apparent that all
students contributed
equally to the project.
Evident that students
worked with each
other in a cooperative
manner and split the
project to match the
student’s strengths.
16
18. Quality Check
Get 10-20 copies of student past work
Sort in to 3 categories of quality (Excellent – Good –
Poor)
Write descriptions for why each piece of work is
categorized
Compare and contrast work to identify relevant criteria
Write descriptions of quality along a continuum for
each criteria
18
25. Conclusion
Recognize common flaws
Describe characteristics of good criteria
Write descriptions that distinguish levels of
performance
Use ExamSoft to assign grades
Ensure rubrics evaluate learning
25
26. Questions?
Contact Information:
Aimee Strang, Pharm.D., MSHPEd
Assistant Dean of Curricular Assessment
Albany College of Pharmacy and Health Sciences
518-694-7320 | aimee.strang@acphs.edu
www.HPEdassessment.com
26
Notes de l'éditeur
By the end of this presentation, I hope you wil be able to
Im assuming you have experience using them. If not, you are interested in them because your listening today. Either way you know a bit about them.
Rubrics is a coherent set of pre-defined criteria that include descriptions of levels of performance quality.
Help teacher teach - Help coordinate instruction
Help students learn - understand what is expected of them and at which level they are performing
Help students self-improve by comparing their work against the rubric
Improve reliability of rating or scoring student learning by keeping the rater focused on what is important
Great tool IF used ot written correctly. Lots of BAD rubrics out there
From scratch … part 2 is NOT to create .. .good for revision of existing rubrics
Pre-rubric step
My course- Trends in Academic Health. Objectives - Analyze current trends in academic health
Both game and poster- just list the trends, present facts and figures… both are engaging but neither meet the LO analyze
Analyze - Organizes and synthesizes evidence to reveal insightful patterns, differences, or similarities related to focus.
Not yes or No – checklist
Medication choice and counseling – not unidimentional
Describe performance – what would you observe? NOT how would you evaluate the work
(Descriptive NOT judgmental)
Don’t use Excellent, good fair, poor
1-4 – middle is catchall
Choose a medication that addressed all symptoms and met patient needs
Choose med that addressed all sx but was not the best choice (cost, doing etc)
Choose a med that only addressed some sx
Choose wrong medication
Confounding outcomes- measuring more than one outcome without recognizing them as separate skills
Assignment was to create health and wellness education for a target population.
Originality AND Audience
Professional language AND Grammer
Go back to learning objectives… what is the learning goal? :
Provide prevention, intervention and educational strategies for select populations to improve health and wellness-
Assess the healthcare status and needs of a targeted population.
Communicate assertively, persuasively, confidently and clearly.
Rubric used to grade
Pharmacy administration course –assignment was to create a business plan for a new pharmacy
(checklist) no levels of criteria
Effort
Nervousness
Confidence
Language
Grade is an evaluation of compliance with the rubric or assignment directions, not of learning
Work in groups to create a SOAP note using evidence based medicine
Main function is to match performance to a description, not judge it immediately. The description is the BRIDGE between student work and evaluation or judgement of learning.
How many levels should there be – use as many id necessary to describe meaningful differences in performance quality
Minimum proficiency = Passing = 60%
By the end of this presentation, I hope you wil be able to