Learning assessment tools and program evaluations tools aren’t interchangeable. What do satisfaction surveys or self-assessments tell us about student learning? Most people aren’t self-aware and students definitely fit that description. Competency theory suggests an inverse relationship between competence and confidence, but our assessment results show no relationship whatsoever! Our poster will visually demonstrate the difference between conducting learning assessment and program evaluation. We will draw attention to utility of each type of assessment in producing meaningful evidence for the value of our instruction programs. This information is crucial for making strategic decisions about assessment: which studies are producing valid, usable results, and which ones can we afford to give up? From the results of a learning assessment paired with self-evaluation at a business school, we show that there is not much relationship at all between what students think they know, and what they actually know when it comes to research skills. Applying the concepts of competency theory helps us argue that students are not good at judging whether they need research instruction. Seeing this laid out in a graphic format also communicates in a high-impact way that you can’t measure student learning with a satisfaction survey.

1. Competence vs. Confidence: Assessment Knockdown!
tinyurl.com/assessmentknockdown
Self-Evaluation vs. Test Score
20
number of students who got this score
What we learned And who got the In the literature
only perfect score? what they say!
We created an information literacy skills quiz 15
A student who said
Gross, M., & Latham, D. (2009). Undergraduate
(available online) and administered it to 40 “My skills are fair”! ! perceptions of information literacy: Defining,
students. We threw in one question asking ! attaining, and self-assessing skills. College &
them to rate their own research skills. Who said
! Research!Libraries, 70(4), 336-350. doi:VL - 70
“According to! research! performed! in the! domain! of
The students’ self-reporting did not “My skills are excellent”? psychology,! people who are incompetent, particularly! in
A student who only scored areas!in which!people commonly have some orientation,
correlate to their quiz scores at all! 10 tend to believe that their skills are above average
3/10 on our quiz! and!to!overestimate!their!performance on a!skills test.”
Self-report questions do not provide reliable
data for learning assessment... Kruger, J., &"Dunning, D. (1999). Unskilled and
! unaware of it:"How difficulties"in recognizing"one's
! own! incompetence lead to inflated self-
POW! ! assessments. Journal of Personality and Social
! Psychology, 77(6), 1121-34.
5
“We propose that those with limited knowledge in a
Learning vs. Program domain suffer a dual burden: Not only do they reach
mistaken conclusions and make regrettable errors, but
Assessment Evaluation their incompetence robs them of the ability to realize it.”
Learning assessment Program evaluation Ragains, P. (1997). Evaluation of academic librarians'
measures whether measures whether ! instructional performance: Report of a national
0 ! survey. Research Strategies, 15(3), 159-175.
students learned anything your program is 0 1 2 3 4 5 6 7 8 9 10 “Distinctions must be made between efforts to measure
as a result of instruction meeting your goals learning outcomes for students (normally determined by
Examples: tests with right or Examples: satisfaction surveys,
test scores out of 10 test results), the overall instructional program, or the
performance of individual librarians.”
wrong answers, scoring papers course evaluations
I don’t know My skills are poor My skills are fair My skills are good My skills are excellent
with rubrics
Amy R. Hofer, Distance Learning Librarian, Portland State University | Margot Hanson, Web Services Librarian, Golden Gate University
ACRL Annual Conference, Philadelphia PA, April 1, 2011