Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Facilitating feedback processes at scale through personalised support actions

602 vues

Publié le

As education keeps advancing into the era of ubiquitous data availability there are certain challenges that are also increasing. The connection between data and direct improvements or benefit for students in terms of the overall quality of the learning experience is still an area under significant evolution. Learning analytics promises the use of data to improve learning experiences, but bridging the distance between widespread data availability and meaningful, effective and relevant actions informed by this data is still important. The current focus when considering the use of data tends to gravitate towards institutional interventions that target only a subset of the students (e.g. those at risk of dropping a course or abandoning the institution). But the student experience is much more complex and varied.

In this talk we will describe OnTask, a platform and approach to facilitate the connection between data and actions in the context of a learning experience. The framework used by the tool contains a generic architecture to simplify the combination of multiple data sources under a single data structure with an intuitive design of rule-based personalized support actions that can be scaled to large student cohorts. OnTask approaches the problem from the benefits of feedback processes that rely on a conversation between students and instructors at the level of a course.

Publié dans : Formation
  • Login to see the comments

  • Soyez le premier à aimer ceci

Facilitating feedback processes at scale through personalised support actions

  1. 1. Facilitating feedback processes at scale through personalised support actions Abelardo Pardo (@abelardopardo)
 Division of Information Technology, Engineering and the Environment slideshare.net/abelardo_pardo LauraLaRose Open Apereo Conference 5 June 2018, Montreal Canada
  2. 2. Abelardo Pardo Facilitating feedback processes at scale 2 Pedagogically sound Technologically Effective
  3. 3. Abelardo Pardo Facilitating feedback processes at scale 3 Help learners succeed
  4. 4. Abelardo Pardo Facilitating feedback processes at scale 4 PaoloMazzoleni Data Learning analytics
  5. 5. Abelardo Pardo Facilitating feedback processes at scale 5 The allure
 of dashboards PNNLPacificNorthwest • Information density • Assumed expertise
  6. 6. Abelardo Pardo Facilitating feedback processes at scale 6 Thaler, R. H., & Sunstein, C. R. (2008). Nudge. Great Britain: Yale University Press. “People make good choices in contexts in which they have experience, good information, and prompt feedback” DerekBruffflickr.com
  7. 7. Abelardo Pardo Facilitating feedback processes at scale 7 Paul A. Kirschner, Mirjam Neelen, https://3starlearningexperiences.wordpress.com/2018/06/05/no-feedback-no-learning/
  8. 8. Abelardo Pardo Facilitating feedback processes at scale 8 Krause, K.-L., Hartley, R., James, R., & McInnis, C. (2005). The First Year Experience in Australian Universities: Findings from a decade of National Studies. University of Melbourne: Centre for the Study of Higher Education. Eleafflickr.com The feedback question gets systematically low values in student surveys
  9. 9. Abelardo Pardo Facilitating feedback processes at scale 9 Liu, D. Y.-T., Bartimote-Aufflick, K., Pardo, A., & Bridgeman, A. J. (2016). Data-driven Personalization of Student Learning Support in Higher Education. In A. Peña-Ayala (Ed.), Learning analytics: Fundaments, applications, and trends: A view of the current state of the art. In preparation: Springer. • Upload student listing • Upload indicators per student • Write filters or rules • Contact selected students with personalised email Student Relationship Engagement System (SRES)
  10. 10. Abelardo Pardo Facilitating feedback processes at scale 10 Report to students Kahn, I., & Pardo, A. (2016). Data2U: Scalable Real time Student Feedback in Active Learning Environments. Paper presented at the International Conference on Learning Analytics and Knowledge, Edinburgh, UK. doi:10.1145/2883851.2883911
  11. 11. Abelardo Pardo Facilitating feedback processes at scale 11 Calsidyrose • Direct connection with data • Intuitive operations for academics • Immediate sense of purpose • Solving a clearly identified pedagogical problem • Aware of wealth of data sources • LMS agnostic — Ecosystem • Support human and machine integration • Extensible with upcoming AI results
  12. 12. Abelardo Pardo Facilitating feedback processes at scale 12 ontasklearning.org
  13. 13. Abelardo Pardo Facilitating feedback processes at scale 13 Data Table Data Frame Data frame to SQL CSV to data frame Data frame operations Pandas Web Application Admin site Security features API implementation Easy to scale to high load MVP and scalable Solid ORM Django
  14. 14. Abelardo Pardo Facilitating feedback processes at scale 14 SimonClancy Quick user tour Three roles • Administrator • Instructor • Learner
  15. 15. Abelardo Pardo Facilitating feedback processes at scale 15 • Users manage a set of workflow • Workflow is data + actions • Fully functional admin pages • Manage configuration options, users, etc. • System-wide connections • Allow generic data import
  16. 16. Abelardo Pardo Facilitating feedback processes at scale 16 • Export workflow • Share with other users • Clone to replicate • Global attributes • Key (unique) columns • Automatically identified Automatic type detection
  17. 17. Abelardo Pardo Facilitating feedback processes at scale 17 • Download CSV file • Allow “Views"
  18. 18. Abelardo Pardo Facilitating feedback processes at scale 18 Arbitrary subsets of rows and columns
  19. 19. Abelardo Pardo Facilitating feedback processes at scale 19
  20. 20. Abelardo Pardo Facilitating feedback processes at scale 20 • Personalised content • Data capture Import actions, promote sharing
  21. 21. Abelardo Pardo Facilitating feedback processes at scale 21 • Select a subset of learners • Arbitrary expression on any column value
  22. 22. Abelardo Pardo Facilitating feedback processes at scale 22 Columns in data table
  23. 23. Abelardo Pardo Facilitating feedback processes at scale 23 • Select a subset of learners • Arbitrary expression on any column value Conditions to include/ignore text
  24. 24. Abelardo Pardo Facilitating feedback processes at scale 24
  25. 25. Abelardo Pardo Facilitating feedback processes at scale 25 • Select a subset of learners • Arbitrary expression on any column value Replace by column value Conditions to include/ignore text Text included only if condition No_Video_1 is truePreview the content for each learner
  26. 26. Abelardo Pardo Facilitating feedback processes at scale 26 Replace by column value Personalised text
  27. 27. Abelardo Pardo Facilitating feedback processes at scale 27 • Send summary email to instructor • Track email reading • Download/export workflow snapshot
  28. 28. Abelardo Pardo Facilitating feedback processes at scale 28
  29. 29. Abelardo Pardo Facilitating feedback processes at scale 29 Include in LMS Time window availability Offer content through URL
  30. 30. Abelardo Pardo Facilitating feedback processes at scale 30 Data capture actions
  31. 31. Abelardo Pardo Facilitating feedback processes at scale 31 Description Subset of learners Questions
  32. 32. Abelardo Pardo Facilitating feedback processes at scale 32 New question Allowed values
  33. 33. Abelardo Pardo Facilitating feedback processes at scale 33 New question
  34. 34. Abelardo Pardo Facilitating feedback processes at scale 34
  35. 35. Abelardo Pardo Facilitating feedback processes at scale 35 Instructor lead data entry
  36. 36. Abelardo Pardo Facilitating feedback processes at scale 36 Select learner
  37. 37. Abelardo Pardo Facilitating feedback processes at scale 37 Enter data
  38. 38. Abelardo Pardo Facilitating feedback processes at scale 38 Schedule Email Messages
  39. 39. Abelardo Pardo Facilitating feedback processes at scale 39
  40. 40. Abelardo Pardo Facilitating feedback processes at scale 40 Obtain data and proceed to merge
  41. 41. Abelardo Pardo Facilitating feedback processes at scale 41 Match keys to merge Choose merge version
  42. 42. Abelardo Pardo Facilitating feedback processes at scale 42 Basic visualisation
  43. 43. Abelardo Pardo Facilitating feedback processes at scale 43
  44. 44. Abelardo Pardo Facilitating feedback processes at scale 44 Basic visualisation
  45. 45. Abelardo Pardo Facilitating feedback processes at scale 45 Value with respect to population
  46. 46. Abelardo Pardo Facilitating feedback processes at scale 46 TaymTaym Quick technical tour
  47. 47. Abelardo Pardo Facilitating feedback processes at scale 47
  48. 48. Abelardo Pardo Facilitating feedback processes at scale 48 Calsidyrose • Several authentication options (SAML2) • LTI authentication • Connection to corporate SMTP server • User identification through email address • Execution of admin scripts through Django shell (ORM capable) • PostgreSQL (tighter integration with django) • Plugins to apply arbitrary transformations Deployment
  49. 49. Abelardo Pardo Facilitating feedback processes at scale 49 • DRF (Django Rest Framework) • Token Authentication
  50. 50. Abelardo Pardo Facilitating feedback processes at scale 50 EdenBrackstone Does it work?
  51. 51. Abelardo Pardo Facilitating feedback processes at scale 51 Helpful feedback Effect size (Cohen’s d) = 0.49 Medium positive effect Midterm Scores Effect size (Cohen’s d) = 0.21 Small positive effect Pardo, A., Jovanović, J., Dawson, S., Gašević, D., & Mirriahi, N. (In press). Using Learning Analytics to Scale the Provision of Personalised Feedback. British Journal of Educational Technology. doi:10.1111/bjet.12592
  52. 52. Abelardo Pardo Facilitating feedback processes at scale 52 Focus groups • “It helps me to validate where I am; do I need to freak out right now?” • “…gives you a nudge — Stop procrastinating and playing games!” • A reminder to study “across the board” (flow-on effect) • “The wording makes you want to do it. Like an encouragement.”
  53. 53. Abelardo Pardo Facilitating feedback processes at scale 53 • Support instructors to manage personalised feedback processes • Simple rule-base knowledge encoding • Provide appropriate view of data sources • Scale to large and highly diverse cohorts • Open-source project • Pilots in 2018 finishing • Demo servers available ontasklearning.org
  54. 54. Abelardo Pardo Facilitating feedback processes at scale 54 Meet (a subset of) the team
  55. 55. Facilitating feedback processes at scale through personalised support actions Abelardo Pardo (@abelardopardo)
 Division of Information Technology, Engineering and the Environment slideshare.net/abelardo_pardo LauraLaRose Open Apereo Conference 5 June 2018, Montreal Canada

×