TESTA workshop intro at Trinity College Dublin

1. Why a programme view?
Why TESTA?
Professor Tansy Jessop
TESTA Workshop
Trinity College Dublin
9 February 2017

2. Jottings
1. One thing you already know about TESTA
2. One problem you have faced with assessment
3. One problem you have faced with feedback
4. One blue sky idea to address a problem

3. What I am hoping to achieve today
1. Brief overview of TESTA
2. Why people find it useful
3. Three problems TESTA addresses
4. Four themes in the data
5. Solutions: a taster

5. Mixed Methods approach
Programme
Team
Meeting
Assessment
Experience
Questionnaire
(AEQ)
TESTA
Programme
Audit
Student
Focus Groups

6. Sustained growth

7. TESTA….
“…is a way of thinking
about assessment and
feedback”
Graham Gibbs

8. It enables you to see the whole elephant

9. Three problemsThree problems
Problem 1: Something awry not sure why
Problem 2: Curriculum design problem
Problem 3: The problem of educational change

10. 1. Something awry, not sure why

11. Wow! Our students
love History! Fantastic!

12. Whoops there’s a little
problem here

13. Fix it!
Ok, we’ll look especially at polishing
up our feedback. Students seems to
find that the least best thing.

14. Apply spit and polish

15. Anyone for the feedback sandwich?
I cushion the
blow!
The hard truths are nicely
disguised!
Me too - nice and
soft!

16. Problem 2: Curriculum design problem

17. Does IKEA 101 work for complex learning?

18. Curriculum privileges ‘knowing’ stuff
“Content is often the most visible aspect
for students, the control of which is
frequently devolved to individual
academics, who receive little or no training
in curriculum design and planning”
(Blackmore and Kandiko 2014, 7).

19. Blunt instrument curriculum

20. Problem 3: Educational change problem
Three misguided assumptions:
1. There is not enough high
quality data.
2. Data will do it
3. Academics will buy it.
http://www.liberalarts.wabash.edu/study-overview/

21. Proving is different from improving
“It is incredibly difficult to translate assessment
evidence into improvements in student learning”
“It’s far less risky and complicated to analyze data
than it is to act”
(Blaich & Wise, 2011)

22. Paradigm What it looks like
Technical rational Focus on data and tools
Relational Focus on people
Emancipatory Focus on systems and structures

23. TESTA themes and impacts
1. Variations in assessment patterns
2. High summative: low formative
3. Disconnected feedback
4. Lack of clarity about goals and standards

24. Defining the terms
• Summative assessment carries a grade which
counts toward the degree classification.
• Formative assessment does not count
towards the degree (either pass/fail or a
grade), elicits comments and is required to be
done by all students.

25. 1. Huge variations
• What is striking for
you about this data?
• How does it compare
with your context?
• Does variation
matter?

26. Assessment features across a 3 year UG degree (n=73)
Characteristic Range
Summative 12 -227
Formative 0 - 116
Varieties of assessment 5 - 21
Proportion of examinations 0% - 87%
Time to return marks & feedback 10 - 42 days
Volume of oral feedback 37 -1800 minutes
Volume of written feedback 936 - 22,000 words

27. Theme 2: High summative: low formative
• Summative ‘pedagogies of control’
• Circa 2 per module in UK
• Ratio of 1:8 of formative to summative
• Formative weakly understood and practised

28. Assessment Arms Race

29. What students say about high summative
• A lot of people don’t do wider reading. You just focus
on your essay question.
• In Weeks 9 to 12 there is hardly anyone in our
lectures. I'd rather use those two hours of lectures to
get the assignment done.
• It’s been non-stop assignments, and I’m now free of
assignments until the exams – I’ve had to rush every
piece of work I’ve done.

30. What students say about formative
• If there are no actual consequences of not
doing it, most students are going to sit in the
bar.
• The lecturers do formative assessment but we
don’t get any feedback on it.

31. Actions based on evidence
1. Rebalance summative and formative
2. Programme approach
3. Formative in the public domain
4. Linking formative and summative
5. Risky, creative, challenging tasks
6. Students reading and producing more
7. Deeper understanding of value of formative

32. Theme 3: Disconnected feedback

33. Take five
• Choose a quote that
strikes you.
• What is the key issue?
• What strategies might
address this issue?

34. What students say…
It’s difficult because your assignments are so detached
from the next one you do for that subject. They don’t
relate to each other.
Because it’s at the end of the module, it doesn’t feed into
our future work.
Because they have to mark so many that our essay
becomes lost in the sea that they have to mark.
It was like ‘Who’s Holly?’ It’s that relationship where
you’re just a student.

35. Actions based on evidence
• Conversation: who starts the dialogue?
• Iterative cycles of reflection across modules
• Quick generic feedback: the ‘Sherlock’ factor
• Feedback synthesis tasks
• Technology: audio, screencast and blogging
• From feedback as ‘telling’…
• … to feedback as asking questions

36. Theme 4: Confusion about goals and
standards
• Consistently low scores on the AEQ for clear
goals and standards
• Alienation from the tools, especially criteria
and guidelines
• Symptoms: perceptions of marker variation,
unfair standards and inconsistencies in practice

37. What students say…
We’ve got two tutors- one marks completely differently to
the other and it’s pot luck which one you get.
They have different criteria, they build up their own
criteria.
It’s such a guessing game.... You don’t know what they
expect from you.
They read the essay and then they get a general
impression, then they pluck a mark from the air.

38. Taking action: internalising goals and
standards
• Regular calibration exercises
• Discussion and dialogue
• Discipline specific criteria (no cut and paste)
Lecturers
• Rewrite/co-create criteria
• Marking exercises
• Design and value formative
Lecturers
and students
• Enter secret garden - peer review
• Engage in drafting processes
• Self-reflection
Students

39. From this educational paradigm…

40. Transmission Model

41. Social Constructivist Model

42. References
Blaich, C., & Wise, K. (2011). From Gathering to Using Assessment Results: Lessons from the Wabash
National Study. Occasional Paper #8. University of Illinois: National Institution for Learning Outcomes
Assessment.
Boud, D. and Molloy, E. (2013) ‘Rethinking models of feedback for learning: The challenge of
design’, Assessment & Evaluation in Higher Education, 38(6), pp. 698–712. doi:
10.1080/02602938.2012.691462.
Gibbs, G. & Simpson, C. (2004) Conditions r which assessment supports students' learning. Learning and
Teaching in Higher Education. 1(1): 3-31.
Harland, T., McLean, A., Wass, R., Miller, E. and Sim, K. N. (2014) ‘An assessment arms race and its
fallout: High-stakes grading and the case for slow scholarship’, Assessment & Evaluation in Higher
Jessop, T. and Tomas, C. 2016 The implications of programme assessment on student learning.
Assessment and Evaluation in Higher Education. Published online 2 August 2016.
Jessop, T. and Maleckar, B. (2014). The Influence of disciplinary assessment patterns on student
learning: a comparative study. Studies in Higher Education. Published Online 27 August 2014
http://www.tandfonline.com/doi/abs/10.1080/03075079.2014.943170
Jessop, T. , El Hakim, Y. and Gibbs, G. (2014) The whole is greater than the sum of its parts: a large-scale
study of students’ learning in response to different assessment patterns. Assessment and Evaluation in
Higher Education. 39(1) 73-88.
Nicol, D. (2010) From monologue to dialogue: improving written feedback processes in mass higher
education, Assessment & Evaluation in Higher Education, 35: 5, 501 – 517.
O'Donovan, B , Price, M. and Rust, C. (2008) 'Developing student understanding of assessment
standards: a nested hierarchy of approaches', Teaching in Higher Education, 13: 2, 205 — 217
Sadler, D. R. (1989) ‘Formative assessment and the design of instructional systems’, Instructional
Science, 18(2), pp. 119–144. doi: 10.1007/bf00117714.