Presentation delivered by Simon Cotterill to the JISC Netskills workshop on Effective Practice with e-Portfolios on 24th June 2010

1. An Introduction to the ePET e-Portfolio and lessons learned along the way… Simon Cotterill Senior Research Associate School of Medical Sciences Education Development Newcastle University http://www.eportfolios.ac.uk

2. Overview Backgroundand Overview of ePET Case Studies: Medicine at Newcastle EPICS-2: blog linked to skills Dental ePortfolio Dynamic Learning Maps Lessons Learned Interoperability

3. A long history…. Newcastle-Nottingham Internet Personal & Academic Records x 2 1998 - present FDTL-4 Project JISC T&L Fund Projects x 2 Postgrad.Research Training ePortfolio Teaching Fellowship ePortfolio for appraisal Postgraduate eProgression Institutional roll-out project ePET project FDTL Transferability Project Mini project (Bioscience) Mini Projects (Medicine) EPICS regional ePortfolioproject EPICS-2 regional ePortfolioproject Mobile portfolio projects PIOP projects x 3 eDoctoring portfolio (California) Dynamic Learning Maps NARN network Dental ePortfolios Hosted services

4. Embedding in the curriculum = better engagement and effective use of ePortfolios Driessen E et al. Portfolios in medical education: why do they meet with mixed success? A systematic review. Med Educ. 2007;41:1224-33. Atlay, M. Embedding PDP practice in the curriculum, in Personal Development Planning and Employability, Higher Education Academy, York. 2006 Cotterill SJ, Horner P, Teasdale D et al. Effective embedding and integration of ePortfolios in medical and dental curricula. IJCOS in press Cotterill SJ, Lowing K, Cain K, Lofthouse R, et al. Blogs and e-Portfolios: can they support reflection, evidencing and dialogue in teacher training. Journal of Learning Development in Higher Education, 2, 2010. … but, what do we mean by embedding?

5. Atlay (2009) classified ways in which PDP can be associated with curricula:

7. Integrated action planning

8. Uploading filesePET: A flexible component-based ePortfolio Opportunity to customise portfolio for specific programmes: Cotterill SJ, Horner P, Hammond GR, et al. Implementing ePortfolios: adapting technology to suit pedagogy and not vice versa ! Proc. ePortfolio 2005

9. Case Studies

10. Medicine: Portfolios to support F2F Annual Appraisals Cotterill SJ, Bradley PM, Stacy R. Using ePortfolios to support annual appraisal in undergraduate medicine. Proc. ‘ePortfolios, identity and personalised learning in healthcare education’, HE Academy MEDEV Subject Centre, Newcastle, 2008. pp 57-63. ISBN 978-1-905788-66-2

12. useful for reflection, interaction/discussion & planning.

13. instructive on how to evidence their development for future appraisals/assessment

14. Students really appreciated personal interaction in the appraisal

15. 58% felt it would influence their learning in Yr 2

16.  Some unclear on purpose/rationale

17.  Discussion too focussed on portfolio process – for some

19. Summative Assessment: ePortfolio for SSC Placements  “It encouraged me to really give thought to what I wanted to achieve during the [SSC] ....As a result of the portfolio I think I got much more out of the [SSC] than I would have otherwise.”  “Most of the things I learned couldn't be 'measured/quantified' so I felt it was of little value to try and invent a way in which they could.”  “It made me concentrate on creating aims at the start of the [SSC] and allowed me to plan the [SSC] with my supervisor in a defined way. Overall it made my learning for the [SSC] more organised and focused.” Printed output from the ePortfolio (partial) Cotterill SJ, Bradley PM, Hammond GR ePortfolios: Supporting assessment in complex educational environments. In: Bryan C and Clegg K, ed. Innovative Assessment in Higher Education. London: Taylor and Francis Group Ltd, 2006.

20. EPICS-2 North East regional collaboration for personalised, work-based, and life-long learning October 2007 to February 2009 Partners include: 5 HEIs FE colleges (Comport Project) CETL4HealthNE Project Director: Prof. Geoff Hammond Project Manager: Simon Cotterill Project Officers: Paul Horner Martin Edney http://www.epics.ac.uk

21. ePET: Unstructured Blogbut with explicit links to skills/outcomes Cotterill SJ, Horner P, Gill S, et al. Beyond the Blog: getting the right level of structure in an ePortfolio to support learning. Proc. ePortfolios 2007, Maastricht

23. Modest engagement: av. 28 logins per student in 18 months (range 1 -168)

24. Minimal engagement prior to assessed assignment in Semester 2, Year 1

25. Mostly used CV/skills (assessed),limited use of cross-module learning log  “It helped me see the skills I was using that my modules shared and sort of helped me develop those further”.  “too time consuming”  “I had to write a CV and it helped me think about what modules and what aspects of each module I enjoy, while writing about them”.  “didn't really understand what I needed to put down as evidence”

27. Medium engagement: av. 30 logins per student (range 2 -142)

28. Steady use over time; recording clinical goals, placements and clinical skills

29. Virtually no sharing of blog entries in the community areas “I find I do not have enough time to use the ePortfolio regularly.”  “being able to put in my placement goals, and have a format to review my progress in these at each stage”.  “Ability to put in different placements into the portfolio alongside your goals and whether you achieved them. Ability to make blogs private to evaluate yourself without others seeing.”.  “You had to be careful to tick a load of boxes saying that your blog was private because it could get posted in the community blog otherwise”

31. High engagement: av. 41 logins per student (range 4 -178) in Semester 1, 1096 files uploaded

32. Good use of blog: av. 16 entries. Many linked to Skills av. 76 links to QTS standards

33. Many blogs published to community areas: 825 entries, 262 comments “good for staying in touch whilst on teaching practice”.  “What I do like about the ePortfolio is that it is designed for the purpose of building a skills repertoire, and allows you to connect thoughts and experiences to the Key Skills.”  “It's not very clear exactly what parts of it are mandatory and what parts are optional”  “the ePortfolio is well set-out with regards to being able to link blogs to the skills pages. The ePortfolio in a sense guides you through the necessaries”.  “It is tedious having to fill in a weekly blog especially when I have many other things to do.”

34. Use of Social Networking and perceptions of its use in learning 91% of respondents use social networking sites (58% on a daily basis). Use of these sites was predominantly for social reasons, though: 62% sometimes used them to communicate with classmates about course-related topics 9% had used them to communicate with teaching staff about course-related topics “…most people see Facebook etc. as an escape from work and it really should stay that way.” “It must never be enforced. Emphasis on ‘social’ networking. Informality is key.” BSc Speech & Language Sciences student Combined Studies student “I prefer to keep social networking sites for personal use and for engaging in general conversation about essays etc in a non-official/non-university domain where it's friends discussing a course. Professional dialogue, opinions on educational matters, lesson plans, theory discussions I prefer to engage in face to face or via the official, nominated online spaces.” PGCE student n=163 (30% response rate)

35. Factors related to engagement with ePortfolio / blog Questionnaire results n=163 (30% response rate, 3 subjects – non-medical) EPICS-2 Project http://www.epics.ac.uk

37. Students record procedures & grade their own performance on clinic

38. Staff grade students’ performance & feedback

40. Supports both self assessment & evaluation/feedback from staff

41. Now used by 430 dental undergraduates & their supervisors/tutors

42. Integration

43. Used on a day-to-day basis at the point of learning

44. Supports assessment across much of the curriculum

45. High levels of engagement & highly valued by students and staffSynchronous & asynchronous access Ellis J, Teasdale DW, Vernazza C, Cotterill SJ, Drummond P, Moss JD, Scott L, Thomason M. Effective use of ePortfolios and mobile technologies to support learning and assessment in Dentistry.Proc.'ePortfolios, identity and personalised learning in healthcare education', HE Academy MEDEV Sub 2008 Vernazza C, Ellis, J, Teasdale D, Cotterill S, Scott L, Thomason M, Drummond P, Moss J. Introduction of an e-portfolio in clinical dentistry: Staff and student views. Eur J Dent Education 2011;15:36-41

46. Dynamic Learning Maps http://learning-maps.ncl.ac.uk Add notes and reflections in any topic in a curriculum map -Linked with ePortfolio using latest standards

47. Lessons Learned

49. Students

50. Staff

51. Perceived value – what’s in it for me?

52. Assessment? Careers / employability?

53. Embed & relate to the curriculum

54. not a ‘bolt on’

55. Training

56. A brief mention at induction is unlikely to be enough

57. Set a specific task early on

58. Plan in a check-point

60. Life Long Learning(interoperability)

62. EUROPASS

63. IMS ePortfolio

64. LEAP2a http://www.leapspecs.net

65. Extended the work in EPICS-2 into 5 JISC-CETIS collaborative projects:

66. PIOP1..3 (completed)

67. XCRI mini project (completed)

68. Competencies (ongoing)Horner P, Cotterill SJ. Meeting the ePortfolio Interoperability Challenge. Proc. ePortfolios, identity and personalised learning in healthcare education. 2008 Newcastle Horner P, Cotterill SJ, Peterson J, Skelly G. Interoperability in Action – ePortfolios, Leap2a and Dynamic Learning Maps Proc. London Learning Forum; ePortfolio – Key Competencies – Identity, 2010, London.

71. Integration with other systemse.g. Workshop admin portfolio/CV e.g. CV recruitment services

73. Novel approach to link blogs to structured outcomes in ePET

74. Work on interoperability standards for lifelong learning helped shape national standards.