1) Leicester Medical School implemented formative e-assessments using ExamSoft during their GI module to provide students with regular feedback. Students took weekly session-by-session exams and four additional exams during the Christmas break.
2) Research found that the formative assessments encouraged students to study earlier and uncover trouble spots. This led students to seek solutions and engage in an active feedback loop to improve their learning.
3) The assessments also pushed students to think and learn at higher levels of Bloom's Taxonomy, such as evaluation and analysis. Students created their own study materials in response to the feedback.
4) Most students completed the in-class exams and over 75% did at least
An Evaluation of Medical Students' Responses to Structured Exam Feedback from Formative E-Assessments
1. An Evaluation of Medical
Students’ Responses to
Structured Exam Feedback
from Formative E-Assessments
Terese Bird, Educational Designer, Leicester Medical School
ASME Scientific Meeting 6 Jul 2016
Belfast, Northern Ireland, UK
3. Groupwork Sessions: Learning by Enquiry
(Cox et al., 2010)
• Students divided into Belbin groups
• Same groups meet for every module
• Groups work on questions in iPad-
workbooks
• Answers given the week after or never
• Sound process of thinking through to
the answer is the point
• Junior doctors are “expert guides”
4. Student study issues
• Lecture notes +
workbook “answers”
= holy canon of
notes on iPad
• “Revision” happens
in the weeks before
exams
• Concern that study
is mainly
“remember”
6. Feedback issues
• Feedback is single highest-impact factor in learning (Rowntree,
1987).
• NSS feedback scores
• End of term is too late
Our approach:
“Little and often” formative assessments via ExamSoft on
iPads
11. Research questions
• Do students who engage with the regular formative feedback in the
GI unit change their study strategy as a result, and if so in what
way(s)?
• Do they look at other material they would not have looked at if they hadn’t used the
exams?
• What new study techniques, if any, did they employ?
• Do students who engage with regular formative exam feedback
improve learning in “thinking levels” associated with higher elements
of Bloom’s Taxonomy?
• Does student engagement with regular formative exam feedback
enable students’ learning benefits in ways that do not occur when the
regular formative exams are not given?
• Do students perceive learning benefits enough to engage with structured formative
12. Examsoft formatives Gastro-Intestinal
Unit Autumn 2015
Exam Type When Content Circumstance
Session-by-session
exams
Weekly, from Oct
through Dec 2015
Material studied in
the past week
Exam done in
groupwork class
sessions, not
under exam
conditions
Christmas 1
Christmas 2
New Year 1
New Year 2
During Christmas
break Dec ember
2015–Jan 2016
All questions from
session-by-session
exams, put
together and
randomised
Four separate
exams with the
same exam
content, offered 4
times throughout
Christmas break
for self-study
whenever students
wanted.
14. How did the formatives change students’
study if at all?
• “A mini-revision before the proper revision” – forced earlier
study
• Uncovered trouble spots which caused them to seek solutions
• Back-and-forth communication = feedback loop
• Active study
15. Thinking and learning on higher levels of
Bloom’s Taxonomy
• If questions were written higher on Bloom’s,
students learnt higher on Bloom’s – constructive
alignment (Biggs, 1996)
• Single Best Answer led to Evaluation and
Analysis
• Students reacted with Create to using exams in
revision
• Created document of info on uncovered trouble spots
• Created answers when they tested themselves without
the multiple-choice
“I’m looking for… an
explanation for why an
answer is the answer. It’s
Single Best Answer, there are
two that are right, but one’s
a better answer.”
16. Student engagement with formative
assessments
• 85% of students completed exams in class
• 77% of students did at least one outside-of-class exam
• Students engaged despite the fact these exams contributed
nothing to their official marks
17. “It’s not clear that I performed
better on summatives because
of these exams, as the
summatives are integrated, but
they helped me to learn GI
better.”
Key words used by students
describing the formative
exams in module evaluation
Winter 2016
18. Recommendations
• Learning objectives and assessments should constructively
align “higher on the Bloom’s” (Biggs, 1996)
• Explore ways encouraging students to act on feedback received
• Advise students regarding their own study:
• Active study
• Little and often
• Throughout the term
• Less dependence on writing “definitive” sets of notes
19. References
• Biggs, J. (1996) ‘Enhancing teaching through constructive alignment’,
Higher Education, [online] Available from:
http://edukologija.vdu.lt/en/system/files/ConstrutivismAligment_Biggs_
96.pdf.
• Cox, B., Calder, M. and Fien, J. (2010) Teaching and Learning for a
Sustainable Future, [online] Available from:
http://www.unesco.org/education/tlsf/mods/theme_d/mod23.html.
• Rowntree, D. (1987) Assessing Students: How shall we know them?,
Revised Ed. London, Kogan Page.
Questions? Email me t.bird@le.ac.uk
@tbirdcymru on Twitter
