Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
@DrBartRienties
Professor of Learning Analytics
What is the impact? Six years of learning
analytics lessons at the Open Un...
What we have learned in six years at the OU
Change is slow, but can be enhanced with:
1. Clear senior management support
2...
Enabling data certainty Precision Education
"Precision education requires two equally important conditions:
accurate predi...
So how was I doing on this ride?
So how am I doing on in the last three months?
1. My dispositions/emotions/drive were substantially different (in
each of these pictures/days)
2. None of these dispositi...
Predictive analytics and professional development
Kuzilek, J., Hlosta, M., Herrmannova, D., Zdrahal, Z., & Wolff, A. (2015...
Start
FS
F RFS
OFS ORF
N O S
RF
R
OF OR ORS
ORFS OS RS
FS
F RFS
OFS ORF
N O S
RF
R
OF OR ORS
ORFS OS RS
FS
F RFS
OFS ORF
N...
FS
F RFS
OFS ORF
N O S
RF
R
OF OR ORS
ORFS OS RS
Start
FS
F RFS
OFS ORF
N O S
RF
R
OF OR ORS
ORFS OS RS
FS
F RFS
OFS ORF
N...
FS
F RFS
OFS ORF
N O S
RF
R
OF OR ORS
ORFS OS RS
Start
FS
F RFS
OFS ORF
N O S
RF
R
OF OR ORS
ORFS OS RS
FS
F RFS
OFS ORF
N...
Probabilistic model: all students
time
TMA1
VLE
start
OU Analyse demo http://analyse.kmi.open.ac.uk
Herodotou, C., Rienties, B., Hlosta, M., Boroowa, A., Mangafa, C., Zdrahal, Z., (2020). Scalable implementation of predict...
Herodotou, C., Rienties, B., Hlosta, M., Boroowa, A., Mangafa, C., Zdrahal, Z., (2020). Scalable implementation of predict...
Student Facing
Analytics
So what is missing here?
Magic of learning design (does not come easy)
“Research on the relationship between learning design and learning
analytics...
McAndrew, P., Nadolski, R. and Little, A., 2005. Developing an approach for Learning Design Players. Journal of Interactiv...
Assimilative Finding and
handling
information
Communication Productive Experiential Interactive/
Adaptive
Assessment
Type ...
Merging big data sets
• Learning design data (>300 modules mapped)
• VLE data
• >140 modules aggregated individual data we...
Nguyen, Q., Rienties, B., Toetenel, L., Ferguson, R., Whitelock, D. (2017). Examining the designs of computer-based assess...
Constructivist
Learning Design
Assessment
Learning Design
Productive
Learning Design
Socio-construct.
Learning Design
VLE ...
http://www.open.ac.uk/blogs/learning-design/
What have I learned in six years at the OU
Change is slow, but can be enhanced with:
1. Clear senior management support
2....
Further reflections
1. Who owns the data?
2. What about the ethics?
3. What about professional development?
4. Are we opti...
References: see oro.open.ac.uk/
• Conole, G. (2012). Designing for Learning in an Open World [Book]. Springer.
• Ferguson,...
T: drBartRienties
E: bart.rienties@open.ac.uk
W: www.bartrienties.nl
W: https://www.organdonation.nhs.uk/
W: https://www.s...
@DrBartRienties
Professor of Learning Analytics
What is the impact? Six years of learning
analytics lessons at the Open Un...
Keynote Data Matters JISC What is the impact? Six years of learning analytics lessons at the Open University
Keynote Data Matters JISC What is the impact? Six years of learning analytics lessons at the Open University
Keynote Data Matters JISC What is the impact? Six years of learning analytics lessons at the Open University
Keynote Data Matters JISC What is the impact? Six years of learning analytics lessons at the Open University
Keynote Data Matters JISC What is the impact? Six years of learning analytics lessons at the Open University
Keynote Data Matters JISC What is the impact? Six years of learning analytics lessons at the Open University
Keynote Data Matters JISC What is the impact? Six years of learning analytics lessons at the Open University
Keynote Data Matters JISC What is the impact? Six years of learning analytics lessons at the Open University
Prochain SlideShare
Chargement dans…5
×

Keynote Data Matters JISC What is the impact? Six years of learning analytics lessons at the Open University

The Open University (OU) was an early adopter of learning analytics, and after six years has had the opportunity to reflect on the impact of large scale adoption across the institution.

Has there been an impact on student retention/progress/completion?
How are the positives (or negatives) reflected in student satisfaction surveys?
What worked, what didn't, and with this benefit of hindsight what is, or should be, next?

  • Soyez le premier à commenter

Keynote Data Matters JISC What is the impact? Six years of learning analytics lessons at the Open University

  1. 1. @DrBartRienties Professor of Learning Analytics What is the impact? Six years of learning analytics lessons at the Open University Speaker Keynote JISC 27 January 2021
  2. 2. What we have learned in six years at the OU Change is slow, but can be enhanced with: 1. Clear senior management support 2. Bottom-up support from teachers and researchers who are willing to take a risk 3. Evidence-based research can gradually change perspectives and narratives 4. You quickly forget about the small/medium/large successes and fail to realise that you are making a real impact 5. Large-scale innovation takes substantial time and effort 6. It is all about people… https://www.jisc.ac.uk/blog/lessons-learned-from-six-years-of-learning-analytics-at-the-open-university-18-jan- 2021
  3. 3. Enabling data certainty Precision Education "Precision education requires two equally important conditions: accurate predictions of academic performance based on early observations of the learning process and the availability of relevant educational intervention options… In our latest publication, “we make a plea for applying dispositional learning analytics (DLA) to make LA precise and actionable“ But is this realistic? Tempelaar, D., Rienties, B., Nguyen, Q. (2021). The contribution of dispositional learning analytics to precision education. Journal of Educational Technology & Society, 24 (1), 109-122.
  4. 4. So how was I doing on this ride?
  5. 5. So how am I doing on in the last three months?
  6. 6. 1. My dispositions/emotions/drive were substantially different (in each of these pictures/days) 2. None of these dispositions are captured accurately in any predictions of these cycling apps 3. These dispositions significantly influence “performance” (80% of cycling performance is mental) 4. What does all this “enabling” data mean for certainty for HE (as most institutions will only collect < 1% of these kinds of data)
  7. 7. Predictive analytics and professional development Kuzilek, J., Hlosta, M., Herrmannova, D., Zdrahal, Z., & Wolff, A. (2015). OU Analyse: analysing at-risk students at The Open University LACE Learning Analytics Review (Vol. LAK15-1). Milton Keynes: Open University. Kuzilek, J., Hlosta, M., & Zdrahal, Z. (2017). Open University Learning Analytics dataset. Scientific Data, 4, 170171. doi: 10.1038/sdata.2017.171 Wolff, A., Zdrahal, Z., Herrmannova, D., Kuzilek, J., & Hlosta, M. (2014). Developing predictive models for early detection of at-risk students on distance learning modules, Workshop: Machine Learning and Learning Analytics Paper presented at the Learning Analytics and Knowledge (2014), Indianapolis.
  8. 8. Start FS F RFS OFS ORF N O S RF R OF OR ORS ORFS OS RS FS F RFS OFS ORF N O S RF R OF OR ORS ORFS OS RS FS F RFS OFS ORF N O S RF R OF OR ORS ORFS OS RS FS F RFS OFS ORF N O S RF R OF OR ORS ORFS OS RS FS F RFS OFS ORF N O S RF R OF OR ORS ORFS OS RS Pass Fail No submit TMA-1 time VLE opens Start Activity space FS F RFS OFS ORF N O S RF R OF OR ORS ORFS OS RS
  9. 9. FS F RFS OFS ORF N O S RF R OF OR ORS ORFS OS RS Start FS F RFS OFS ORF N O S RF R OF OR ORS ORFS OS RS FS F RFS OFS ORF N O S RF R OF OR ORS ORFS OS RS FS F RFS OFS ORF N O S RF R OF OR ORS ORFS OS RS FS F RFS OFS ORF N O S RF R OF OR ORS ORFS OS RS Pass Fail No submit TMA-1 time VLE opens Start VLE trail: successful student FS F RFS OFS ORF N O S RF R OF OR ORS ORFS OS RS
  10. 10. FS F RFS OFS ORF N O S RF R OF OR ORS ORFS OS RS Start FS F RFS OFS ORF N O S RF R OF OR ORS ORFS OS RS FS F RFS OFS ORF N O S RF R OF OR ORS ORFS OS RS FS F RFS OFS ORF N O S RF R OF OR ORS ORFS OS RS FS F RFS OFS ORF N O S RF R OF OR ORS ORFS OS RS FS F RFS OFS ORF N O S RF R OF OR ORS ORFS OS RS Pass Fail No submit TMA-1 time VLE opens Start VLE trail: student who did not submit
  11. 11. Probabilistic model: all students time TMA1 VLE start
  12. 12. OU Analyse demo http://analyse.kmi.open.ac.uk
  13. 13. Herodotou, C., Rienties, B., Hlosta, M., Boroowa, A., Mangafa, C., Zdrahal, Z., (2020). Scalable implementation of predictive learning analytics at a distance learning university: Insights from a longitudinal case study. Internet and Higher Education, 45, 100725.
  14. 14. Herodotou, C., Rienties, B., Hlosta, M., Boroowa, A., Mangafa, C., Zdrahal, Z., (2020). Scalable implementation of predictive learning analytics at a distance learning university: Insights from a longitudinal case study. Internet and Higher Education, 45, 100725. Amongst the factors shown to be critical to the scalable PLA implementation were: Faculty's engagement with OUA, teachers as “champions”, evidence generation and dissemination, digital literacy, and conceptions about teaching online.
  15. 15. Student Facing Analytics So what is missing here?
  16. 16. Magic of learning design (does not come easy) “Research on the relationship between learning design and learning analytics has also been a focus in European research in recent years. For example, in their research at the Open University UK, Toetenel and Rienties combine learning design and learning analytics where learning design provides context to empirical data about OU courses enabling the learning analytics to give insight into learning design decisions. This research is important as it attempts to close the virtuous cycle between learning design to improve courses and enhancing the quality of learning, something that has been lacking in the research literature. For example, they study the impact of learning design on pedagogical decision-making and on future course design, and the relationship between learning design and student behaviour and outcomes (Toetenel and Rienties 2016; Rienties and Toetenel 2016; Rienties et al. 2015).” Wasson, B., & Kirschner, P. A. (2020). Learning Design: European Approaches. TechTrends, 1-13.
  17. 17. McAndrew, P., Nadolski, R. and Little, A., 2005. Developing an approach for Learning Design Players. Journal of Interactive Media in Education, 2005(1), p.Art. 15. DOI: http://doi.org/10.5334/2005-14
  18. 18. Assimilative Finding and handling information Communication Productive Experiential Interactive/ Adaptive Assessment Type of activity Attending to information Searching for and processing information Discussing module related content with at least one other person (student or tutor) Actively constructing an artefact Applying learning in a real-world setting Applying learning in a simulated setting All forms of assessment, whether continuous, end of module, or formative (assessment for learning) Examples of activity Read, Watch, Listen, Think about, Access, Observe, Review, Study List, Analyse, Collate, Plot, Find, Discover, Access, Use, Gather, Order, Classify, Select, Assess, Manipulate Communicate, Debate, Discuss, Argue, Share, Report, Collaborate, Present, Describe, Question Create, Build, Make, Design, Construct, Contribute, Complete, Produce, Write, Draw, Refine, Compose, Synthesise, Remix Practice, Apply, Mimic, Experience, Explore, Investigate, Perform, Engage Explore, Experiment, Trial, Improve, Model, Simulate Write, Present, Report, Demonstrate, Critique Conole, G. (2012). Designing for Learning in an Open World. Dordrecht: Springer. Rienties, B., Toetenel, L., (2016). The impact of learning design on student behaviour, satisfaction and performance: a cross-institutional comparison across 151 modules. Computers in Human Behavior, 60 (2016), 333-341 Open University Learning Design Initiative (OULDI)
  19. 19. Merging big data sets • Learning design data (>300 modules mapped) • VLE data • >140 modules aggregated individual data weekly • >37 modules individual fine-grained data daily • Student feedback data (>140) • Academic Performance (>140) • Predictive analytics data (>40) • Data sets merged and cleaned • 111,256 students undertook these modules
  20. 20. Nguyen, Q., Rienties, B., Toetenel, L., Ferguson, R., Whitelock, D. (2017). Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates. Computers in Human Behavior. DOI: 10.1016/j.chb.2017.03.028. 69% of what students are doing in a week is determined by us, teachers!
  21. 21. Constructivist Learning Design Assessment Learning Design Productive Learning Design Socio-construct. Learning Design VLE Engagement Student Satisfaction Student retention 150+ modules Week 1 Week 2 Week30 + Rienties, B., Toetenel, L., (2016). The impact of learning design on student behaviour, satisfaction and performance: a cross-institutional comparison across 151 modules. Computers in Human Behavior, 60 (2016), 333-341 Nguyen, Q., Rienties, B., Toetenel, L., Ferguson, R., Whitelock, D. (2017). Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates. Computers in Human Behavior. DOI: 10.1016/j.chb.2017.03.028. Communication
  22. 22. http://www.open.ac.uk/blogs/learning-design/
  23. 23. What have I learned in six years at the OU Change is slow, but can be enhanced with: 1. Clear senior management support 2. Bottom-up support from teachers and researchers who are willing to take a risk 3. Evidence-based research can gradually change perspectives and narratives 4. You quickly forget about the small/medium/large successes and fail to realise that you are making a real impact 5. Large-scale innovation takes substantial time and effort 6. It is all about people…
  24. 24. Further reflections 1. Who owns the data? 2. What about the ethics? 3. What about professional development? 4. Are we optimising the record player?
  25. 25. References: see oro.open.ac.uk/ • Conole, G. (2012). Designing for Learning in an Open World [Book]. Springer. • Ferguson, R., & Buckingham Shum, S. (2012). Social learning analytics: five approaches. 2nd International Conference on Learning Analytics and Knowledge, Vancouver, British Columbia, Canada. • Herodotou, C., Rienties, B., Hlosta, M., Boroowa, A., Mangafa, C., & Zdrahal, Z. (2020). The scalable implementation of predictive learning analytics at a distance learning university: Insights from a longitudinal case study. The Internet and Higher Education, 45, 100725. https://doi.org/10.1016/j.iheduc.2020.100725 • Kuzilek, J., Hlosta, M., Herrmannova, D., Zdrahal, Z., & Wolff, A. (2015). OU Analyse: analysing at-risk students at The Open University. Learning Analytics Review, 1-16. http://oro.open.ac.uk/42529/1/__userdata_documents5_ajj375_Desktop_analysing-at-risk-students-at-open-university.pdf • McAndrew, P., Nadolski, R., & & Little, A. (2005). Developing an approach for Learning Design Players. Journal of Interactive Media in Education, 2005(1). https://doi.org/10.5334/2005-14 • Nguyen, Q., Rienties, B., Toetenel, L., Ferguson, F., & Whitelock, D. (2017). Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates. Computers in Human Behavior, 76(November 2017), 703-714. https://doi.org/10.1016/j.chb.2017.03.028 • Rienties, B., Olney, T., Nichols, M., & Herodotou, C. (2020). Effective usage of Learning Analytics: What do practitioners want and where should distance learning institutions be going? Open Learning, 35(2), 178-195. • Rienties, B., & Toetenel, L. (2016). The impact of learning design on student behaviour, satisfaction and performance: a cross-institutional comparison across 151 modules. Computers in Human Behavior, 60, 333-341. https://doi.org/10.1016/j.chb.2016.02.074 • Toetenel, L., & Rienties, B. (2016). Analysing 157 Learning Designs using Learning Analytic approaches as a means to evaluate the impact of pedagogical decision-making. British Journal of Educational Technology, 47(5), 981–992. https://doi.org/10.1111/bjet.12423 • Wakelam, E., Jefferies, A., Davey, N., & Sun, Y. (2019). The potential for student performance prediction in small cohorts with minimal available attributes. British Journal of Educational Technology, 0(0). https://doi.org/10.1111/bjet.12836 • Wasson, B., & Kirschner, P. A. (2020). Learning Design: European Approaches. TechTrends, 1-13. https://link.springer.com/content/pdf/10.1007/s11528-020- 00498-0.pdf • Wolff, A., Zdrahal, Z., Nikolov, A., & Pantucek, M. (2013). Improving retention: predicting at-risk students by analysing clicking behaviour in a virtual learning environment. Proceedings of the Third International Conference on Learning Analytics and Knowledge, Indianapolis.
  26. 26. T: drBartRienties E: bart.rienties@open.ac.uk W: www.bartrienties.nl W: https://www.organdonation.nhs.uk/ W: https://www.sportentransplantatie.nl/
  27. 27. @DrBartRienties Professor of Learning Analytics What is the impact? Six years of learning analytics lessons at the Open University Speaker Keynote JISC 27 January 2021

×