Implementation presentation of InterACT May25 2012

1. CENTRE FOR MEDICAL EDUCATION
interACT – interactive assessment
and collaboration via technology
Rola Ajjawi, Susie Schofield, Karen Barton, Sean McAleer,
Grant Murray
Centre for Medical Education, University of Dundee
Lorraine Walsh, David Walker
Library and Learning Centre, University of Dundee

2. Context
 Postgraduate Certificate, Diploma and Master’s in
Medical Education
 Longstanding internationally renowned course, distance
and paper-based
 2010-2012 curriculum review
 Online delivery
 Larger modules (15 cps)
 Update content
 Criterion referenced F/P/D

3. The Postgraduate Certificate The Diploma will equip The Masters will equip
will equip practitioners to be practitioners to be leaders in practitioners to be educational
teachers driven by a strong medical education and will researchers utilising rigorous
pedagogical and evidence-based consist of an additional 4 and diverse methodologies to
approach. modules – 2 core answer innovative and original
It will consist of 4 modules – 2 (“Curriculum Planning in questions. It will consist of
core (“Teaching and Learning in Medical Education” and either: a 10,000-15,000 word
Medical Education” and “The “Management and dissertation or a research based
Principles of Assessment in Leadership in Medical journal article (45 credits) plus
Medical Education”) + 2 options Education”) + 2 options an additional optional module
http://www.dundee.ac.uk/meded/courses/

4. Student profile
 3000 students, 550 average new enrolments per year
 Primarily doctors but some health professionals including
nursing and dentistry
 75% UK-based, remainder global
 Non-cohort based, no deadlines (flexible start and finish
dates)

5. Challenges of current feedback model
 monologic culture of feedback
 lack of learner engagement with feedback
 lack of understanding of feedback
 transmitted feedback creates dependency on teacher
 no evidence of feedforward
 not utilising self and peer feedback
 lack of a shared context for assessment for teacher and
learner
 high teacher effort— low efficiency
 reduced staff satisfaction as developmental potential of
feedback not seen

6. Aim
The overall aim of this project is to improve feedback dialogue
in an ODL postgraduate medical education programme
1. a longitudinal self-reflective feedforward system
developed within Blackboard
2. a toolkit of e-activities designed to improve self, peer and
tutor dialogic feedback through the use of available
technologies such as blogs, Friendfeed, Skype/webinars, or
synchronous chat

7. Baseline activities
 Document analysis: external examiner reports (2006-
2011); end of course evaluations (2006-2011); additional
evaluation surveys conducted in 2010 and 2011 as part of
our curriculum review; and the HEA postgraduate student
experience survey
 Meeting with staff including outmarkers
 Currently doing a feedback audit of 2011

8. Baseline activities revealed
1. Inconsistency in the quality and quantity of feedback
provided
2. Assessment design (e.g. over-assessment, over-
reliance on essays, lack of formative assessment)
3. Timeliness of the feedback
4. Lack of assessment and feedback dialogue

9. Outcome measures
 Attrition rates: based on graduation per year
 Rates of progress through the module: manual obtained from
BB
 Student survey of perceptions/experiences of feedback:
annual February
 Relevant data from regular module evaluation
 Relevant data from external examiner reports
 Qualitative focus groups: students, academic staff and admin
 Feedback audit: sample size needed?
 Tool to measure reflective gain ??
 Community engagement

10. http://www.screencast.com/t/fsELBHbWuy8C

12. Generic principles
Assessment for learning should:
1. Support personalised learning
2. Ensure feedback leads to learning (i.e. feedforward)
3. Focus on student self-development
4. Stimulate dialogue
5. Induct students in the educational culture and
assessment and feedback practices
6. Be acceptable to all stakeholders

13. Effective feedback dialogue:
 involves a transactional approach to communication
 is jointly owned
 is based on self-evaluation
 comprises negotiation of meaning
 is a two-way developmental process
 is a learning experience

14. Proposed changes – phase I
(09/11-08/12)
• Student submits assignment with self-evaluation
• Student receives feedback on assignment word
document
• Student uploads assignment document into wiki
• Reflects on feedback/answers questions
• Tutor comments on student reflection

15. Step 1 – cover page
Content (understanding of theory / Self-evaluation:
principles and application to own Teaching and Learning in Medical Education -
context)
Assignment 1 - Cover Page
- Understanding of learning theory
Name: Matriculation No:
- Evidence of critical reflection on
learning theory and key learning and Tutor feedback:
teaching principles
Date: Email Address:
- Evidence of application of learning
theory to own practice
Please write a short evaluation of how well your
assignment addresses the criteria listed:
Style, format and language (e.g. Self-evaluation:
structure, coherence, flow, formatting,
use of language)
Tutor feedback:
Sources and references (e.g. range of Self-evaluation:
references cited, relevance, consistency,
accuracy and completeness of
referencing)
Tutor feedback:
Which aspect(s) of your assignment Student comment:
would you specifically like feedback on?
Tutor feedback:
How did previous feedback inform this Student comment:
assignment?
Name of Tutor: Date:

16. Reflection on feedback Wiki
questions
1. How well does the tutor feedback match with your self-
evaluation?
2. What did you learn from the feedback process?
3. What actions, if any, will you take in response to the
feedback process?
4. What if anything is unclear about the tutor feedback?

19. I thought it was pleasing to hear at
yesterday's seminar that Neil McPherson
had found significant benefits of
students having the ability to access
previous feedback on-screen - bodes
well for this project.
http://www.slideshare.net/neilgmcpherson/inspired-university-of-dundee-23-may-2012

20. How does student know who to share with?

21. Challenge
 How is list
populated?
 What happens
if 2+ people
same name?
 What happens
if tutor has
different name
(John McAleer
known as
Sean)

22. How does tutor know when a wiki is updated?

23. Alert by email

24. Challenge
 Numbers (35
per month)
 Searchable?
 Sortable?

26. Advantages Challenges
 across programme  student and staff
 open to individual student engagement
and staff  alerts to staff and students
 staff can see all feedback  technology clunkiness
provided during programme  administrative issues with
 promotes dialogue about returned assignments
feedback
 opportunity to
calibrate/benchmark
 instant feedback to marker!

27. Proposed changes – stage II
(09/12-08/13)
 Develop a toolkit of e-activities designed to improve self,
peer and tutor dialogic feedback through the use of
available technologies such as blogs, Friendfeed,
Skype/webinars, or synchronous chat

28. Project website
http://blog.dundee.ac.uk/interact

29. Thank you
interACT is a JISC-funded project
interACT@dundee.ac.uk