Visit to a blind student's school🧑🦯🧑🦯(community medicine)
Developing Self-regulated Learning in High-school Students: The Role of Learning Analytics
1. Developing self-regulated learning in high-school
students: The role of learning analytics
Vitomir Kovanovic, Srecko Joksimovic
The Centre for Change and Complexity in Learning (C3L)
University of South Australia
Wayne Jaeschke
Trinity College
2. Outline
▰ Trinity College Context & current SRL development
▰ Future of Learning at Trinity College
▰ The Role of Learning Analytics in SRL development
▰ LA and the Attainment of Graduate Qualities
▰ Workshop Activity- LA and SRL
2
3. Trinity College by School
School # Students #Teachers ICSEA
Blakeview 639 44 1030
Gawler River 758 55 1024
North 838 63 1052
South 835 62 1052
Senior 619 54
Total 3689 278 1041
https://www.youtube.com/watch?time_continue=1&v=U2BxrNV60xk
3
6. Trinity College - University & VET Pathways
6
▰ ~300 Year 12
graduates each
year
▰ 70% University
applicants
7. Trinity College - Graduate Performance at
University (Challenge #1)
7
▰ University of South Australia Report (September 2016)
▰ Old Scholar Survey Results (3 years post graduation)
8. Trinity College - Implementation of LMS
8
▰ Implementation of LMS (Schoolbox) at Senior School in 2017
▰ Implementation across middle schools in 2018
(Year 10 only year level with devices)
9. Trinity College - Digitisation of Current Practices
& Drowning in Data (Challenge #2)
9
▰ The Feedback imperative
▰ Staff workload associated with digitisation of existing practices- ‘Effort’
rubric
▻ 12 indicators * 27 students per class * 11 classes * 8 reporting
periods per year = 28,512 rubric clicks
10. Trinity College - The ‘Effort’ Rubric
10
Challenges 1 Managing challenges
2 Sharing work
Communication 3 Active Listening
4 Learning conversations
5 Questioning
Feedback 6 Utilising feedback
Learning from Mistakes 7 Managing mistakes
8 Self-reflection
Organisation 9 Completion of assessment tasks and homework
10 Punctuality and preparedness
Strategies for Learning 11 Collaboration
12 Perseverance
11. The Future of Education - Influences
11
▰ Melbourne Declaration
▰ Through Growth to Achievement - Gonski
▰ OECD Education 2030
12. The Future of Education - Trinity College Learning
Outcomes
12
▰ Students who are self-regulated & lifelong
learners
▰ Students who demonstrate attainment of the
general capabilities
13. The Future of Education - Trinity College Common
Intent & Transformation of Education
13
▰ Trinity Institute (Launch January 2019)
▰ Trinity College Graduate Qualities
▰ General Capabilities embedded in learning activities
▰ Problem and Project Based Learning
▰ LMS & LA to provide appropriate, timely and
actionable feedback to all stakeholders
14. LA and the Attainment of GQ at Trinity College
14
▰ What aspects of the current reporting and feedback process are
valuable and should be maintained?
▰ What indicators of student learning and behaviour could be
passively generated from LMS data?
▰ What aspects of the current processes seem to be redundant or
of little value and should be discarded or adjusted?
▰ How can the large volume of data be refined into feedback that is
timely and actionable for students, teachers, managers and
parents?
▰ What assessments and student activities can be adjusted to
generate meaningful data?
15. What can analytics be used for?
▰ What students are at risk?
▻ What are the indicators of risk?
▻ What trends we see with at-risk students?
▰ What students engage with?
▰ Which interventions are most effective?
15
16. What we can measure with analytics?
▰ How active are the students?
▻ When they log to the system?
▻ How frequently?
▰ What resources are accessing?
▻ Frequency of access?
▻ Timing of access?
▻ Amount of access?
▰ Do students engage with feedback?
▻ Do they read it
▻ Do they act on it? 16
17. Self-regulated learning (SRL)
▰ Ed Psychology theory of how students learn and adjust they
approach to learning
▻ Students are agents
▻ Students adjust they learning approach
▻ Often not so well
▻ Often stuck to the same approach
17
20. How is SRL assessed?
20
▰ Think-aloud protocols
▰ Self-reports
▻ MSLQ: Motivated Strategies for Learning Questionnaire
▰ LMS trace data
▰ Multimodal data capture
▻ Cameras & Facial recognition tools
▻ Microphones & NLP techniques
▻ RFID tags (Social network analysis)
21. MSLQ (Motivated Strategies for Learning
Questionnaire) components
21
▰ Motivation
▻ Intrinsic goal orientation
▻ Extrinsic goal orientation
▻ Task value
▰ Expectancy
▻ Control beliefs
▻ Self-efficacy beliefs
▰ Affect & Anxiety
▰ Cognitive & Metacognitive Strategies
▻ Rehearsal
▻ Elaboration
▻ Organization
▻ Critical Thinking
▻ Metacognitive Self-Regulation
▰ Resource Management Strategies
▻ Time and Study Environment
▻ Effort Regulation
▻ Peer Learning
▻ Help Seeking
22. MSLQ challenges
22
▰ Research instrument
▰ Too long: 81 items (some shorter alternatives available)
▰ Can’t be used continuously
▰ Reliability
▻ Only 27% correlation between self-reports & trace data
23. Learning Analytics for SRL
23
Pros
▰ Automated capturing
▰ Continuous monitoring
Cons
▰ Cannot capture all SRL processes
▻ Goal of learning design
▰ Too much context dependent
▻ Need for robust measures
24. Learning Analytics for SRL
24
▰ Need for standardized measures
▻ Different stakeholders
▻ Different course contexts
25. Learning Design and Learning Analytics
▰ Learning activities must be created with analytics in mind
▻ What processes activities should promote?
▻ What data will be collected?
▰ Data capturing should not drive learning design
▻ Should be unobtrusive
▻ Simple for end-users
25
26. Evidence-centered assessment design
▰ Assessment development method
▻ Student model: What to measure?
▻ Evidence model: How to measure?
▻ Task model: When to measure?
26
27. MSLQ (Motivated Strategies for Learning
Questionnaire) components
27
▰ Motivation
▻ Intrinsic goal orientation
▻ Extrinsic goal orientation
▻ Task value
▰ Expectancy
▻ Control beliefs
▻ Self-efficacy beliefs
▰ Affect & Anxiety
▰ Cognitive & Metacognitive Strategies
▻ Rehearsal
▻ Elaboration
▻ Organization
▻ Critical Thinking
▻ Metacognitive Self-Regulation
▰ Resource Management Strategies
▻ Time and Study Environment
▻ Effort Regulation
▻ Peer Learning
▻ Help Seeking
28. Approach #1
▰ Student model: What to measure?
▻ Goal setting & goal orientation
▻ Rehearsal
▻ Elaboration
▻ Organisation
▻ Critical thinking
▻ Time management
▻ Peer learning
▻ Help seeking
28
29. Approach #1
▰ Evidence model: How to measure?
▻ Goal setting: survey
▻ Rehearsal: LMS traces
▻ Elaboration: LMS traces
▻ Organisation: LMS traces
▻ Critical thinking: Text analysis
▻ Time management: Attendance & LMS traces
▻ Peer learning: LMS discussions
▻ Help seeking: LMS discussions
29
30. Approach #1
▰ Task model: When to measure?
▻ Goal setting: survey
▻ Rehearsal: LMS traces
▻ Elaboration: LMS traces
▻ Organisation: LMS traces
▻ Critical thinking: Text analysis
▻ Time management: Attendance & LMS traces
▻ Peer learning: LMS discussions
▻ Help seeking: LMS discussions
30
31. LA approach for fostering SRL
▰ Adjusting the design of single/few core course(s) to promote SRL
strategies e.g.
▻ Formative LMS quizzes (rehearsal)
▻ Essay outlining activity (organisation)
▻ Creating annotated bibliography (elaboration)
▰ Develop metrics & models to capture SRL strategies
▻ Multiple threshold metrics
▻ IRT-based analytics models
▰ Provide students with feedback based on model output
▻ Capture reflections on provided feedback & suggestions
31
32. Issues with the proposed approach
▰ SRL is not course-specific
▰ SRL processes not full captured by LMS data
▰ SRL more fine-grained than what is provided by trace logs
▰ Use of technology primarily to enable data collection
▰ How to affect student motivation & beliefs?
▰ Primary and middle school students do not use LMS extensively
32
33. New opportunities
▰ Classroom data-capturing
▻ Multimodal data capturing tools
▻ Video recording
▻ Conversation recording
▻ Digital learning tools (e.g., tablets)
33
37. What's next?
- 3 questions
- 3x10 minutes
- 7 minutes preparation within the group
- 3 minutes public discussion
- Each group
- Open a Google doc
- Have fun...
37
38. Workshop Activity - Designing for Feedback
38
▰ Identify stakeholders and their role in contributing to students’
development of SRL
39. Workshop Activity - Designing for Feedback
39
▰ What insights do the stakeholders need?
40. Workshop Activity - Designing for Feedback
40
▰ How can pertinent data be captured to gain the requisite insights?
41. Workshop Activity - Designing for Feedback
41
▰ How should insights be communicated to each stakeholder?