Presentation by Dr Ann Ooms and Hendrik van der Sluis, Kingston University, at the "Improving Assessment and Feedback Practices in a Technology-Enhanced Teaching and Learning Environment: Theory and Practice" Event, 19th May 2010 at Kingston University. Part of the "Higher Education Academy : Evidence Based Practice Seminar Series 2010"
The presentation provides an overview of recent literature concerning assessment and feedback
Mattingly "AI & Prompt Design: The Basics of Prompt Design"
Assessment & Feedback Literature Review
1. Improving Assessment and Feedback Practices in a Technology-Enhanced Teaching and Learning Environment: An Overview of the Literature Dr Ann Ooms Faculty of Health and Social Care Sciences Kingston University and St. George’s University of London Hendrik van der Sluis Academic Development Centre Kingston University
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
Notes de l'éditeur
National student survey 2009 Assessment and feedback “ The 2009 results show that 24 of the 52 courses had fewer than 50% of students who agreed that, ‘feedback on my work has helped me clarify things I didn’t understand’” (National student survey 2009: 4) Promptness of feedback “ 20 courses had fewer than 50% of students who agreed that, ‘feedback has been prompt’. This correlates with the 2008 NSS results where promptness and clarity were highlighted as problem areas” (National student survey 2009: 4) National student survey 2005-2007 Assessment and Feedback “ Students aged 31 or over on entry to their course were more positive about Assessment and Feedback than students aged 18 on entry to their course. Dyslexic students were less positive than students with no declared disability. Students who entered their course from an Access course were more positive than students who entered with A levels. Students living with parents or guardians were more positive than students in other types of accommodation. Students from outside the UK were more positive than those from the UK. Students in institutions with fewer than 10000 students were more positive than students in institutions with more than 20000 students.” (The National Student Survey 2005-2007: Findings and Trends: 16) National student survey 2006 Results by institution 2006 http://education.guardian.co.uk/students/table/0,,1857547,00.html
More mp3 papers from conference “A Word In Your Ear 2009 - Audio Feedback”, 18/12/2009 at Sheffield Hallam University, available on: http://research.shu.ac.uk/lti/awordinyourear2009/
References Ames, C. (1992). Classrooms: Goals, structures, and student motivation. Journal of Educational Psychology, 84, 3, 261-271. Bangert-Drowns, R.L., Kulick, J.A., and Morgan, M.T. (1991). The instructional effect of feedback in test-like events. Review of Educational Research, 61, 2, 213-238. Bell, B. (1995). Interviewing: A technique for assessing science knowledge. In S. Glynn & R. Duit (Eds.), Learning Science in Schools: research reforming practice. New Jersey: Lawrence Erlbaum Associates. Bell, B. and Cowie, B. (2001). The Characteristics of Formative Assessment in Science Education, Science Education, 85, 536–553. Bennington, A. (2007) Stick It in Your Ear: Keeping Current with Podcasts ONLINE, May 2007 issue, viewed 20 September 2007, http://www.infotoday.com/online/may07/index.shtml Billett, S. (1996) Towards a Model of Workplace Learning: The learning curriculum. Studies in Continuing Education, 18, 1, 43-58. Black, P., and Wiliam, D. (1998a). Assessment and classroom learning. Assessment in Education, 5 (1): 7-74. Black, P. and Wiliam, D. (1998b). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80 (2): 139-148. (Available online: http://www.pdkintl.org/kappan/kbla9810.htm.) Boulos, M, Maramba, I & Wheeler, S (2006). Wikis, blogs and podcasts: a new generation of Web-based tools for virtual collaborative clinical practice and education, BMC Medical Education, vol. 6, available at http://www.biomedcentral.com/1472-6920/6/41, accessed 10 October 2008. Boyle J.T. and Nicol, D.J., (2003). Using classroom communication systems to support interaction and discussion in large class settings, Association for Learning Technology Journal, 11(3), 43-57. Brittain S, Glowacki P, Van Ittersum J, Johnson L. (2006) “Podcasting Lectures.” EDUCAUSE Quarterly, Vol. 29 No. 3, 24–31, available at http://www.educause.edu/ir/library/pdf/eqm0634.pdf, accessed 13 October 2008. Bruner, J. (1960) Acts of Meaning. Cambridge, MS, Harvard University Press. Buchanan, T. (2000). The Efficacy of a World-Wide Web Mediated Formative Assessment. Journal of Computer Assisted Learning, 16 (3), 193-200. Butler, D.L. & Winnie, P.H. (1995) Feedback and self-regulated learning: a theoretical synthesis. Review of Educational Research, 65(3), 245-281. Chappuis, J.(2009). ‘Seven Strategies of Assessment for Learning’. Hawker Brownlow Education Pty Ltd Cue, N. (1998). A universal learning tool for classrooms? Paper presented at the First Quality in Teaching and Learning Conference, Hong Kong. Cicchino, R. and Mirliss, D. (2006) Tablet PCs: A Powerful Teaching Tool, in Tablet PC Research Project: COPHS Mobile Computing Program, M Genovese, compiler). Butler University, available at https://www.butler.edu/ir/cmsExternalDocuments/tabletResearchProject.pdf, accessed 8 October 2008. DiBattista, D. Mitterer, J.O and Gosse, L. (2004) Acceptance by undergraduates of the immediate feedback assessment technique for multiple-choice testing, Teaching in Higher Education, 9, 1, 17 - 28. Dickenson, K (1997). Distance learning on the Internet: Testing students using Web Forms and the computer-gateway interface, TechTrends, 42, 2, 43 – 46. Dirks, M (1998) How is assessment being done in distance learning? Paper presented at the Web98 conference, Northern Arizona University, 28 – 30 May. Draper, S.W. and Brown, M.I., (2004). Increasing interactivity in lectures using and electronic voting systems, Journal of Computer Assisted Learning, 20, 81-94. Elawar, M.C., and Corno, L. (1985). A factorial experiment in teachers' written feedback on student homework: Changing teacher behaviour a little rather than a lot. Journal of Educational Psychology, 77, 2, 162-173. Elwood, J. and Klendowski, V (2002). Creating of shared practice: the challenges of assessment use in learning and teaching, Assessment and Evaluation in Higher Education, 27, 3, 243 – 256. Freeman, M. (1997). Flexibility in access, interaction and assessment: The case for web-based teaching programs, Australian Journal of Educational Technology, 13(1), 23-37. Garrison, C. and Ehringhaus, M. (2004) Formative and Summative Assessments in the Classroom, National Middle School Association, available at nmsa.org/Publications/WebExclusive/Assessment/tabid/1120/Default.aspx, accessed 5 October 2008. Gaytan, J. & McEwen, B.C. (2007) ‘Effective Online Instructional and Assessment Strategies’. The American Journal of Distance Education, 21(3), pp.117–132 Gibbs, G.; Lucas, L.; Simonite, V.(1996) ‘Class Size and Student Performance: 1984-94. Studies in Higher Education, 21(3), pp. 231-73. Gibbs, G. and Simpson, C. (2002) Does your assessment support your students’ learning, Centre for Higher Education Practice, Open University. Gibbs, G. (2006). ‘Why Assessment is changing’. In Bryan, C. & Clegg, K. (2006) ‘Innovative Assessment in Higher Education’ (2nd Ed.). Oxon, Routledge Gibbs, G. (2006). ‘How assessment frames student learning’. In Bryan, C. & Clegg, K. (2006) ‘Innovative Assessment in Higher Education’ (2nd Ed.). Oxon, Routledge Gibbs, G. and Simpson, C. (2004). Conditions under which assessment supports learning, Learning and Teaching in Higher Education, 1, 3 -31. Glaser, R., & Silver, E. (1994). Assessment, testing, and instruction: Retrospect and prospect. Los Angeles, National Center for Research on Evaluation, Standards, and Student Testing (CRESST). Handley, K. & Williams, L. (2009) ‘From copying to learning: using exemplars to engage students with assessment criteria and feedback’. Assessment & Evaluation in Higher Education, , First published on: 02 October 2009 (iFirst), pp. 1–14, Hargreaves, E. (2005). Assessment for learning? Thinking outside the (black) box. Cambridge Journal of Education 35, no. 2: 213–224. HEFCE (2006) National Student Survey, results published in The Guardian, at http://education.guardian.co.uk/students/table/0,,1857547,00.html HEFCE (2008) 'The National Student Survey 2005-2007: Findings and trends'. [Online]. Available from: http://www.hefce.ac.uk/learning/nss/ HEFCE (2010) ‘2009 National Student Survey’. [Online]. Available from: http://www.hefce.ac.uk/learning/nss/ Heinrich, E., Milne, J., Ramsay, A. & Morrison, D. (2010) ‘Recommendations for the use of e-tools for improvements around assignment marking quality’. Assessment & Evaluation in Higher Education, 34(4), pp. 469-79 Higgins, R., Hartley, P. & Skelton, A. (2001). Getting the message across: the problem of communicating assessment feedback. Teaching in Higher Education. 6, 2, 269-274. Hodgen, J., and B. Marshall. (2005). Assessment for learning in English and Mathematics: A comparison. The Curriculum Journal 16, no. 2: 153–176. Hounsell, D. (2007). ‘Towards more sustainable feedback to students’. In Boud, D. & Falchikov, N. (2007). ‘Rethinking Assessment in Higher Education. Learning for the longer term’, (2nd Ed.). Oxon, Routledge Hudson, J.N. and Bristow, D.R. (2006). Formative assessment can be fun as well as educational, Advances in Physiology Education, 30, 33 – 37. Hursh, T. (undated) Tablet PCs for Classroom Use: Technology and Application, available at http://www.ed.uiuc.edu/courses/edpsy317/sp03/challenge-reports/Hursh-TabletPC.html, accessed 6 October 2008. Ivanic, R., Clark, R. and Rimmershaw, R. (2000), What am I supposed to make of this? The messages conveyed to students by tutors’ written comments. In M.R. Lea and B. Stierer, (eds.), Student Writing in Higher Education: New Contexts. Buckingham: SHRE/Open University Press. Juniu S (2003). Implementing wireless technology in the classroom: the iPAQ project. College Planning and Management, 6, 10, p.38-40. Keegan D (2003). The future of learning: From eLearning to mLearning. Hagen: Femstudienforchung, Germany. E-published version available: http://learning.ericsson.net/mlearning2/old_sites/book.html, accessed 8 October 2008. Kerka, S. and Wonacott, M.E. (2000) Assessing learners online, Practitioner File, ERIC Clearinghouse on Adult, Career and Vocational Education. Washington, DS, Office of Educational Research and Improvement. Laurilard, D. (2002) Rethinking University Teaching: A conversational framework for the effective use of learning technologies. 2nd edition. London: Routledge Falmer Laurillard, D.(2008) Digital Technologies and Their Role in Achieving Our Ambitions for Education. Inaugural lecture. [Online]. Available from: http://www.ioe.ac.uk/about/21450.html Keppell, M., Au, E., Ma, A. & Chan, C. (2006) ‘Peer learning and learning-oriented assessment in technology-enhanced Environments’ Assessment & Evaluation in Higher Education, 31(4), pp. 453-464. Knight, P. (2002) ‘Summative Assessment in Higher Education: practices in disarray’. Studies in Higher Education , 27(3), pp. 275-86 Knight, P. (2006) 'The local practices of assessment', Assessment & Evaluation in Higher Education , 31(4), pp. 435-52 Lawson, D. (1999). Formative assessment using computer-aided assessment, Teaching Mathematics and its Applications, 18, 4, 155 – 158. Liang, X. and Creasy, K. (2004) Classroom assessment in web-based instructional environment: Instructors’ experiences, Practical Assessment, Research and Evaluation, 9, 7, available at http://PAREonline.net/getvn?v=9&n=7, accessed 1 October 2008. Lilley, M. and Barker, T. (2007) Students’ perceived usefulness of formative feedback for a computer-adaptive test, Electronic Journal of eLearning, 5, 1, 31 – 38. Masikunas, G., Panayiotidis, A.and Burke, L. (2007) The use of electronic voting systems in lectures within business and marketing: a case study of their impact on student learning, ALT-J, 15, 1, pp. 3–20 Meng, P (2005) Podcasting and Vodcasting: A white paper – definitions, discussions and implications. University of Missouri, available at http://edmarketing.apple.com/adcinstitute/wp-content/Missouri_Podcasting_White_Paper.pdf, accessed 12 October 2008. Miller, Tess(2009) 'Formative computer-based assessment in higher education: the effectiveness of feedback in supporting student learning', Assessment & Evaluation in Higher Education, 34: 2, 181-92. Middleton, A., Nortcliffe, A., Owens, R., (2009) iGather: learners as responsible audio collectors of tutor, peer and self reflection. Paper presented at: A Word In Your Ear 2009 - Audio Feedback, 18/12/2009 at Sheffield Hallam University. [Online] available from: http://research.shu.ac.uk/lti/awordinyourear2009/docs/Middleton-Nortcliffe-Owens-iGather_final.pdf Nicol, D (2008). Transforming Assessment and Feedback: Enhancing Integration and Empowerment in the First Year. Centre for Academic Practice and Learning Enhancement (CAPLE), University of Strathclyde Nicol, D and MacFarlane-Dick, D (2004) Rethinking formative assessment in HE: a theoretical model and seven principles of good feedback practice, in Enhancing student learning through effective formative feedback , York, HEA. Nicol,D.J. & Boyle, J. T. (2003) Peer Instruction versus Class-wide Discussion in large classes: a comparison of two interaction methods in the wired classroom. Studies in Higher Education 28(4), 457-473 Nicol, D.J. & Milligan, C. (2006), Rethinking technology-supported assessment in terms of the seven principles of good feedback practice. In C. Bryan and K. Clegg (Eds), Innovative Assessment in Higher Education, Taylor and Francis Group Ltd, London Nicol, D.J. (2010) ‘Assessment for learner self-regulation: enhancing achievement in the first year using learning technologies’. Assessment & Evaluation in Higher Education , 34(3), pp. 335-52. Orsmond, P., Merry, S,. Reiling, K (2005) Biology students’ utilisation of tutors’ formative feedback: a qualitative interview study, Assessment and Evaluation in Higher Education, 30, 4, 369 – 386. Peat, M., & Franklin, S. (2003). Has Student Learning been Improved by the Use of Online and Offline Formative Assessment Opportunities? Australian Journal of Educational Technology, 19 (1), 87-99. Perry, D. (2003) Handheld computers (PDAs) in Schools. British Educational Communications and Technology Agency, accessed from http://publications.becta.org.uk/display.cfm, on 5 October 2008 Ramaprasad, A. (1983). On the definition of feedback. Behavioral Science, 28 (1): 4-13. Ribchester, C., France, D., and Wheeler, A.. (2007) Podcasting: a tool for enhancing assessment feedback. Proceedings of4th Education in a Changing Environment, pp131 -139 University of Salford. Available online at: http://www.ece.salford.ac.uk/proceedings/papers/15_07.pdf, 23 May 2008 Rodway-Dyer, S.; Dunne, E. & Newcombe, M. (2009) ‘0207 Audio and Screen visual feedback to support student learning’. Conference paper at A Word In Your Ear 2009 - Audio Feedback, 18/12/2010 Sheffield Hallam University. [Online] Available from: http://research.shu.ac.uk/lti/awordinyourear2009/ Rotheram, B. (2007) Using an MP3 recorder to give feedback on student assignments. Educational Developments, Issue 8.2, June, pp.7-10, Staff and Educational Development Association, London. Prins, F.J., Sluijsmans, D.M.A., Kirschner, P.A. & Strijbos, J.W. (2005) ‘Formative peer assessment in a CSCL environment: a case study’. Assessment & Evaluation in Higher Education , 30(4), pp. 417–444. Sadler, D.R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18 (2): 119-144. Sadler, D.R. (1998) ‘Formative Assessment: revisiting the territory’. Assessment in Education: Principles, Policy & Practice, 5(1), pp. 77-84 Savill-Smiith, C. (2005) Colloquium - The use of palmtop computers for learning: a review of the literature. British Journal of Educational Technology, 36, 3, 567 Savill-Smith, C.and Kent, P. (2003). The Use of Palmtop Computers in Learning. London, LSDA. Scheeler, M.C.,Mcafee, J.K. and Ruhl, K. L. (2006) Effects of Corrective Feedback via Wireless Technology on Pre-service Teacher Performance and Student Behavior, Teacher Education and Special Education Journal, 29, 1, 12 – 25. Shepard, L. A. (2000). The role of classroom assessment in teaching and learning. Los Angeles, National Center for Research on Evaluation, Standards, - 568.and Student Testing (CRESST). Smith, T. (2003). Personal digital assistants (PDAs) in further and higher education, London, JISC, accessed from http://www.ts-consulting.co.uk/DownloadDocuments/PDAsinFurther&HigherEducation.doc, on 5 October 2008. Stiggins, R. (1997) Student-Centred Classroom Assessment (2nd edition). Upper Saddle River NJ, Merrill Tilson, T., McLean, G.N. and Warner, J. (1998) An Evaluation of Distance Learning in the University of Minnesota Human Resource Development Program. In Academy of Human Resource Development Conference Proceedings, Oak Brook, IL, 4 – 8 March, (R, Torraco, ed.)., pp.23 – 30. Tinkelman, R. (2006) Podcasting Comes to Med School Curriculum, Web Weekly: News from the Harvard Medical School Community, January, available at http://webweekly.hms.harvard.edu/archive/2006/0130/student_scene.html, accessed 14 October 2008. Vispoel, W.P., and Austin, J.R. (1995). Success and failure in junior high school: A critical incident approach to understanding students' attributional beliefs. American Educational Research Journal, 32, 2, 377-412. Wali, E., Winters, N. and Oliver, M (2008) Maintaining, changing and crossing contexts: an activity theoretic reinterpretation of mobile learning, ALT-J, 16, 1, 41–57. Weavers, C. (2003) What do students want from assessment? Exchange, Issue 4, Spring, available at www.exchange.ac.uk/files/eissue4.pdf, accessed 9 October 2008. Wickstrom, C.D. (2003) ‘A “funny” think happened on the way to the form’. Journal of Adolescent & Adult Literacy, 46(1), pp. 414-23. Wiliam, D., Lee, C., Harrison, C. and Black, P. (2004) Teachers developing assessment for learning: impact on student achievement, Assessment in Education, 11, 1, 49 – 65. Wit, E., (2003). “Who wants to be….The Use of a Personal Response System in Statistics Teaching”, MSOR Connections, 3 (2), 14-20. Yorke M (2003) Formative assessment in higher education: moves towards theory and the enhancement of pedagogic practice. Higher Education 45, 4, 477-501. Yorke, M. (2005). Formative assessment in higher education: Its significance for employability, and steps to its enhancement, Tertiary Education and Management, 11, 219 – 238. Yorke, M.(2007) Assessment, especially in the first year of higher education: old principles in new wrapping? Keynote Paper presented at the 2003 annual REAP online conference, Assessment design for learner responsibility 29-31 May 2007, available at http://www.reap.ac.uk/reap07/Portals/2/CSL/keynotes/mantz%20yorke/Assessment_old_principles_new_wrapping.pdf, accessed 9 October 2008. Yorke M and Longden B (2007) The first-year experience in higher education in the UK: report on Phase 1 of a project funded by the Higher Education Academy. Available at http://www.heacademy.ac.uk/research/FirstYearExperience.pdf, accessed 8 October 2008.