Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Provision of personalized feedback at scale using learning analytics

7 032 vues

Publié le

The increasing presence of technology mediation offers an unprecedented opportunity to use detailed data sets about the interactions that occur while a learning experience is being enacted. Areas such as Learning Analytics or Educational Data Mining have explored numerous algorithms and techniques to process these data sets. Additionally, technology now offers the opportunity to increase the immediacy of interventions. However, not much emphasis has been placed on how to extract truly actionable knowledge and how to bring it effectively as part of a learning experience. In this talk, we will use the concept of feedback as the focus to establish a specific connection between the knowledge derived from data-analysis procedures and the actions that can be immediately deployed in a learning environment. We will discuss how there is a trade-off between low-level automatic feedback and high-level complex feedback and how technology can provide efficient solutions for the case of large or highly diverse cohorts.

Publié dans : Formation
  • Login to see the comments

Provision of personalized feedback at scale using learning analytics

  1. 1. Provision of personalized feedback at scale using learning analytics Centre for the Enhancement of Teaching and Learning Faculty of Education Hong Kong University 19/May/2017 Abelardo Pardo (@abelardopardo)
 Faculty of Engineering and IT slideshare.net/abelardo_pardo Antoniettaflickr.com
  2. 2. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 2 MaureenBarlinflickr.com New
 Design
 Mindset The role of data and feedback Personalised feedback at scale
  3. 3. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 3 MaureenBarlinflickr.com New
 Design
 Mindset
  4. 4. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 4 Simple information transfer is not working Mazur, E. (2009). Farewell, lecture. Science, 323(5910), 50-51. Krugazorflickr.com
  5. 5. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 5
  6. 6. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 6 Active Learning Works Engage students in
 the learning process Bonwell, C. C., & Eison, J. A. (1991). Active Learning: Creating Excitement in the Classroom ASHEERIC Higher Education Report No. 1. Washington, DC, USA: George Washington University.
  7. 7. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 7 “… robust correlations between student involvement in a subset of ‘educationally purposive activities’, and positive outcomes of student success and development, including satisfaction, persistence, academic achievement and social engagement” Trowler, V. (2010). Student engagement literature review. York, UK: The Higher Education Academy. shuaGandersonflickr.com
  8. 8. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 8 Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: beliefs, techniques, and illusions. Annu Rev Psychol, 64, 417-444. doi:10.1146/annurev-psych-113011-143823 ChristianWeidingerflickr.com We, as learners may • Not know how to promote comprehension, retention, transfer. • Not assess properly our own learning • Be biased when judging our learning • Rely too much on social beliefs
  9. 9. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 9 JansonHewsflickr.com Frontier between physical and virtual spaces is blurring
  10. 10. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 10 Beware of technology creating the illusion of rational thinking JenRflickr.com
  11. 11. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 11 “… teaching in higher education will necessarily shift the balance of its efforts towards a greater investment in design as a way of coping with otherwise intolerable pressures on staff and resources.” Goodyear, P. (2015). Teaching as Design. HERDSA Review of Higher Education, 2, 27-50.
  12. 12. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 12 "There is no such thing as a neutral design" JeremyBrooksflickr.com Thaler, R. H., & Sunstein, C. R. (2008). Nudge. Great Britain: Yale University Press. “People make good choices in contexts in which they have experience, good information, and prompt feedback"
  13. 13. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 13 “38 meta-analyses investigating 105 correlates of achievement, based on 3,330 effect sizes from almost 2 million students” Schneider, M., & Preckel, F. (2017). Variables Associated With Achievement in Higher Education: A Systematic Review of Meta- Analyses. Psychological Bulletin. doi:10.1037/bul0000098 • The effectivity of courses is strongly related to what teachers do. • The effectivity of teaching methods depends on how are implemented • Teachers can improve the instructional quality of their courses by making a number of small changes - providing detailed task-focused and improvement-oriented feedback • The combination of teacher-cantered and student-cantered instructional elements is more effective than either form of instruction alone Variables associated with achievement
  14. 14. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 14 MaureenBarlinflickr.com New
 Design
 Mindset The role of data and feedback
  15. 15. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 15 Students are less likely to engage in pre-class activities if they are not interactive, do not provide formative feedback, and not coherently linked with the face-to-face activities O'Flaherty, J., & Phillips, C. (2015). The use of flipped classrooms in higher education: A scoping review. The Internet and Higher Education, 25, 85-95. doi:10.1016/j.iheduc.2015.02.002 DanKlimkeflickr.com
  16. 16. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics If you could choose one… • Over 800 meta-analyses of student achievements • 100 factors with potential influence • Feedback in top five • (74 meta-analyses) Most effective form: video, audio, computer- assisted instructional feedback, and/or related goals 16 Hattie, J. (2008). Visible learning: A synthesis of over 800 meta-analyses related to achievement. New York: Routledge.
  17. 17. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 17 Boud, D., & Molloy, E. (Eds.). (2013). Feedback in Higher and Professional Education: Understanding it and doing it well. London and New York: Routledge. FarukAteşflickr.com “Feedback is a process whereby learners obtain information about their work in order to appreciate the similarities and differences between the appropriate standards for any given work, and the qualities of the work itself, in order to generate improved work”
  18. 18. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 18 Black, P., & Wiliam, D. (1998). Assessment and Classroom Learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7-74. doi:10.1080/0969595980050102 GarethChristopherflickr.com Innovations designed to strengthen the frequent feedback that students receive about their learning yield substantial learning gains
  19. 19. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 19 Hounsell, D. (2007). Toward more sustainable feedback to students. In D. Boud & N. Falchikov (Eds.), Rethinking Assessment in Higher Education: Learning for the Longer Term. London and New York: Routledge. Perceived as an administrative chore 
 instead of a pedagogical necessity MarcinWicharyflickr.com
  20. 20. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 20 IanRansleyflickr.com How to scale sustainable practices? Carless, D., Salter, D., Yang, M., & Lam, J. (2011). Developing sustainable feedback practices. Studies in Higher Education, 36(4), 395-407. doi:10.1080/03075071003642449 • Multi-stage assignments • Dialogic feedback • Technology supported • Self-evaluation
  21. 21. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 21 Baker, R. S. (2016). Stupid Tutoring Systems, Intelligent Humans. International Journal of Artificial Intelligence in Education. doi: 10.1007/s40593-016-0105-0 From artificial intelligence to intelligence amplification Instructors are informed about student engagement
 but it is up to them to decide if/when/how to act Tecnaliaflickr.com
  22. 22. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 22 DmitryGrigorievflickr.com Learning Analytics: measure, collect, analyse data about learners to understand and improve their learning and the environment in which it occurs 1st International Conference on Learning Analytics and Knowledge [Online]. https://tekri.athabascau.ca/analytics. [Accessed 2017].
  23. 23. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 23 • Collect data about how students engage in a learning experience • Interpret the observations in the context of the instructional design • Translate knowledge into personalised student support actions
  24. 24. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 24 Bartimote-Aufflick, K., Reimann P., Pardo, A. The perspective theory brings to learning analytics in the classroom: A realist approach Manuscript in preparation. Learning Analytics Model
  25. 25. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 25 MaureenBarlinflickr.com New
 Design
 Mindset The role of data and feedback Personalised feedback at scale
  26. 26. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 26 Example of Highly Instrumentalized Learning Design
  27. 27. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 27 Weekly real time feedback
  28. 28. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 28 No statistically significant difference in the rating of feedback (2013 edition, M=3.25, SD=0.97; 2014 edition, M=3.35, SD=1.03); t(389.78) = -0.97, p <0.17
  29. 29. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 29 You should take a more careful look at how symbols are encoded in the video. Would you be able to encode/ decode UAL symbols without looking at the video? Good initial work. However, did you understand the trick to handle encoding with a variable number of bits? Would you be able to provide an example? Good work. Would you be able to come up with your own machine language and your encoding scheme? Remember that it has to be unambiguous. Thorough work with the task about machine language encoding. Give it a quick review before the midterm. Q1 Q2 Q3 Q4 Instructor
  30. 30. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 30 Algorithm
  31. 31. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 31 Automatic Email
  32. 32. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 32 Helpful feedback Effect size (Cohen’s d) = 0.49. Medium positive effect Midterm Scores Effect size (Cohen’s d) = 0.21. Small positive effect Pardo, A., Jovanovic, J., Dawson, S., Gašević, D. & Mirriahi, N. (2017). Using Learning Analytics to Scale the Provision of Personalised Feedback. Manuscript in preparation
  33. 33. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 33 ontasklearning.org
  34. 34. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 34 • Support instructors to create personalised feedback • Simple rule-base knowledge encoding • Provide appropriate view of data sources • Scale to large and highly diverse cohorts • Will be released as open-source project Q3/4 2017 • First pilots in Q1/2 2017 • Tutorial in LAK 2017 • Contact us if interested ontasklearning.org
  35. 35. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 35 MTC0316flickr.com Conclusions • New L&T design mindset required • Feedback is effective to promote student engagement • Learning analytics has the potential to support students at scale • Use data to provide personalised student support
  36. 36. Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 36
  37. 37. Provision of personalized feedback at scale using learning analytics Abelardo Pardo (@abelardopardo)
 Faculty of Engineering and IT slideshare.net/abelardo_pardo Antoniettaflickr.com Centre for the Enhancement of Teaching and Learning Faculty of Education Hong Kong University 19/May/2017

×