These slides are from a journal club discussion at the Pedagogic Research in the Biosciences group at the University of Leicester (UK). The meeting was reflecting on the paper "Oral versus written assessments: a test of student performance and attitudes" by Mark Huxham and colleagues from Napier University, Edinburgh. The paper is due to appear in the February 2012 edition of Assessment and Evaluation in Higher Education

1. Bioscience PedR Journal Club, January 2012
Oral versus written assessments: a test
of student performance and attitudes
Mark Huxham, Fiona Campbell & Jenny Westwood
Assessment & Evaluation in Higher Education
37:125-136 (February 2012)
Reflections led by Chris Willmott (cjrw2@le.ac.uk)

2. Why discuss this paper?
• Interesting research question:
- is there a difference between oral v written
assignments in terms of:
(a) student performance
(b) student attitude?
• Relevant research methodologies
- quantitative test scores
- qualitative: email survey, focus groups
• Involves biologists

3. The oral exam (viva voce)
• Has a noble history
• Has largely been replaced by alternative assessments
in undergraduate courses (in the UK )
• Remains central to examination of PhDs (in the UK )

4. Possible benefits of oral assessment
1. Development of oral communication skills
2. More “authentic” assessment
(i.e. relevant to „real life‟ post-graduation)
3. More inclusive (e.g. for dyslexic students)
4. Gauging understanding & Encouraging critical
thinking
5. Less potential for plagiarism
6. Better at conveying nuances of meaning
7. Easier to spot rote-learning

5. Possible issues with oral assessment
1. Take a long time, impossible for n=300 yr 1
2. Reliability and bias across groups
3. Anonymous marking is impossible
4. Difficult to produce evidence for Externals
5. Stressful (for students)
6. Might favour extrovert and confident students
7. Not appropriate for abstract reasoning tasks

6. Aims of the paper (p127)
1. Do results in oral & written exams differ? Does it
make any difference if questions are assessing
(abstract) scientific questions or personal skills?
2. Do students find oral exams more stressful?
3. What do students think are the relative merits of
oral v written assessments?

7. Students involved in study
Three cohorts:
• Evolutionary biology (yr 1 of 4, Scottish system)
n = 99 (72% , 28%  )
• Applied terrestrial ecology (yr 3 of 4, 2008)
n = 29 (21 , 8  )
• Applied terrestrial ecology (yr 3 of 4, 2007)
n = 18 (11 , 7  )

8. Cohort 1: Year 1_08 students (1)
• Formative test
- 7 x SAQ
- understanding, not fact regurgitation
• Allocated randomly to either “oral” or “written”
- “oral” = 15 mins, 1-2-1 with examiner (n=10)
standardised interview protocol & mark sheet
interviewers met a priori re model answers
- “written” = 30 mins, exam conditions
- both tests same questions
• Given 4 weeks notice of allocation, possible to swap
- 2 swapped oral to written (non-English)
- 2 swapped written to oral (dyslexic)

9. Cohort 1: Year 1_08 students (2)
• 7 questions marked 0 to 2:
0 = no answer, completely wrong
1 = partially right
2 = correct, inc all key points
t-test on mean scores
• “oral” – short notes & marked before next candidate
“written” – double-blind marked
• single additional question:
“how nervous were you about taking this test?”
1 = not at all nervous
4 = very nervous
chi-squared test

10. Cohort 1: Year 1_08 students (3)
• n=91 (45 oral, 46 written)
• Overall difference
mean for written = 6.24 (out of 14)
mean for oral = 8.17, i.e 1.93 more
two-sample t-test: t-value = 3.46, df = 89,
p-value = 0.001
• Gender differences
female oral > written 2.03
male oral > written 1.50

11. Cohort 1: Year 1_08 students (4)
• Nervousness
• Students were more nervous about oral exam
• 2 = 6.78, df = 3, p-value = 0.079

12. Cohort 2: Year 3_08 students (1)
• Prior to this study “Applied terrestrial ecology” module
already assessed by 4-question oral examination
2 x “scientific analysis”
2 x “personal and professional development”
Questions tailored to individual students
• For research, addition of 2 written questions
1 x “scientific analysis”
1 x “personal and professional development”
• 8 mins written assessment before 15 mins oral exam

13. Cohort 2: Year 3_08 students (2)
• All questions (oral and written) assessed on 7-point
scale (0 to 6)
• 1/3 of oral exams double-marked (i.e. 2 interviewers)
• All written questions double-marked

14. Cohort 3: Year 3_07 students
• Same 4-question oral examination as in 2008, i.e.
2 x “scientific analysis”
2 x “personal and professional development”
• No written examination

15. Qualitative assessment (1)
• 2007 Yr 3 students = post-test, post-feedback
email to group
• “Please describe how you felt the interview went. In
particular, how did you perform compared to a more
conventional assessment (such as a written exam)?
What do you think the advantages an disadvantages
of being assessed by interview are, and what lessons
you learn from the experience?”
• 15 out of 18 responded

16. Qualitative assessment (2)
• 2008 Yr 1 students = selected students (number
unspecified) invited to attend focus group
• Intention was for focus group to include
representation from diverse populations in cohort:
- male & female
- home & international
- mature students & school leavers
• Sandwich lunch offered
• Conversation recorded, permission for anonymised
quotation obtained
• 3 students participated

17. Qualitative assessment (3)
• 2008 Yr 3 students = selected students (number
unspecified) invited to attend focus group
• Same procedure as for Yr 1 focus group
• 4 students participated
• Recordings from both focus groups transcribed and
subjected to thematic analysis

18. Summary of available data
• Yr 1 2008 cohort
- 99 in group
- 91 took test (either written or oral)
- 3 offered qualitative data (focus group)
• Yr 3 2008 cohort
- 29 in group
- 24 took test (written and oral)
- 4 offered qualitative data (focus group)
• Yr 3 2007 cohort
- ? in group
- 18 took test (oral only)
- 15 offered qualitative data (email survey)

19. Nervousness
• Students reported being more nervous about oral
exams (Yr 1 specific question, Yr 3 emails)
• Are nerves just caused by lack of familiarity?
• Authors note nerves do not necessarily hinder
performance (and may bolster it)
• David Cameron‟s
“full-bladder” technique?

20. What aspect of the paper
were done well/less well?

21. Good aspects of paper
• Posed an interesting and relevant question re
contemporary assessment
• Involved a combination of quantitative and qualitative
data, including more than one cohort and/or year
• Demonstrated a statistically-significant difference
between performance oral > written test
• Included randomised test (Yr 1 either/or) and a
paired test (Yr 3 both/and)

22. Zadie Smith quote
• Character using words „modern‟ and „science‟ “as if
someone had lent him the words and made him
swear not to break them ”
• Reminiscent of final year library project where
student who “owned” language in science chapter
but phrasing of ethics chapter seemed “borrowed”
(as if working in a foreign language)

23. Questions/Concerns about paper
• Smallest cohort disproportionately represented in
qualitative review?
• 2007 cohort emailed responses – not anonymous?
Fear of impact on (future) modules?
• Question to Yr 3 2007 asked about “interview” not
“oral examination”. Might this influence outcome?
• What were the focus group questions? Appendix?
• What themes emerged from “thematic analysis” other
than anxiety?

24. Reflections re our context
• Current Strategic Review of Bioscience – should we
make more use of oral assessments?
• We have questions as part of project seminar, but
what other applications might be appropriate?
• Would a switch to more oral assessment:
- be better for students?
- be better for us?
- give a truer picture of abilities?
- be a capitulation to student whim?
• Oral assessment allows interaction that reveals if
knowledge has depth or is a veneer

25. Acknowledgement & Disclaimer
• These slides are from a presentation to the School of
Biological Sciences Pedagogic Research group at the
University of Leicester, 17th January 2012
• With the exception of slides with a green border (like
this one) all the credit for the content is due to Mark
Huxham and colleagues, authors of the paper under
discussion
Original citation: Mark Huxham, Fiona Campbell & Jenny Westwood
Assessment and Evaluation in Higher Education 37:125-136 (February 2012)