A reflection on where we are with learning analytics as a new multi-discipline research area. Reflections from the Learning Analytics Conference 2013 with respect to Assessment.
1. Learning Analytics and Student Feedback
Professor Denise Whitelock
The Open University, Walton Hall,
Milton Keynes MK7 6AA, UK
denise.whitelock@open.ac.uk
2. Learning Analytics and Student Feedback
• What is Learning
Analytics?
• Origins
• Early work
• Learning Analytics
and Assessment
DMW UOC May 2013
3. Definition
“Learning Analytics are concerned with the measurement,
collection, analysis and reporting of data about learners
and their contexts, for purposes of understanding and
optimising learning and the environments in which it
occurs”.
Reference
SoLAR, Open Learning Analytics: An Integrated &
Modularised Platform, White Paper, Society for Learning
Analytics Research, 2011.
DMW UOC May 2013
5. Middle Space
• 3rd
LAK Conference
• Learning explicit
• New analytic
methods
• Computational
• Representational
• Statistical
• Visualisation
DMW UOC May 2013
6. Can LAK hold together for long?
• Challenges
• Different
methodologies
• Different theories
• Different predjucies
• Agreement on topics 3rd
LAK 2013
• Visualisation, social network
analysis, communication
and collaboration, discourse
analytics, predictive
analytics, sequence
analytics, assessment
After Suthers & Verbert (2013)
DMW UOC May 2013
7. Political and economic drivers
• Educause Review (2007)
• Academic Analytics,
Campbell & Oblinger (2007)
• Large data and stats =
predictive modelling
• Improve number of
graduates in US
• US finding now with school
exam data
• Hand code
• M.L.
• Apply whote set
• Make predictions
DMW UOC May 2013
8. Formative Research towards separate field
• Open Learner models (Bull & Kay, 2007)
• Social Network analysis (De Laat et al, 2007)
• Networks Adapting Pedagogical Practice (SNAPP),
(Dawson et al, 2010)
• Visualisation of large data sets, Honeycomb (van Ham
et al, 2009)
• Gephi: open source tool (Bastian et al, 2009)
• Signals (Arnold, 2010)
DMW UOC May 2013
9. Signals: Flagship Project
• Moves data from VLE
• Combines with prediction
models
• Real time red/amber/green
traffic lights
• Pilot study (Arnold, 2010)
showed
• Students sought help
earlier
• 12% more B/C grades
• 14% less D/F grades
DMW UOC May 2013
10. Formative Assessment and Learning
Analytics (1)
Tempelaar et al, 2013
• 1st
year Math & Stats
undergraduates,
Maastricht
• Reason text book
• Online questions
• Practice and
performance tests
• 92% higher practice
pass
• 51% lower practice pass
DMW UOC May 2013
11. Formative Assessment and Learning
Analytics (2)
Important for SAFeSEA?
• Learning styles (Vermunt, 1996)
• Self regulation for deep learning
• Practice for stepwise learning
• Motivation and engagement wheel (Martin,
2007)
• Learning emotions
• Pekrun’s control-value theory of learning
emotions
DMW UOC May 2013
12. Performance of video lectures
• Findings from Mirriahi &
Dawson (2013)
• Correlations between
quizzes and lectures
• Shows misalignment
between Assessment
and Teaching
materials?
DMW UOC May 2013
13. HOU2LEARN
• PLE from Hellenic Open
University
• Social network analysis
and final grades
• Online collaboration is
not a predictor for final
grade
• Koulocheri & Xenos,
2013
DMW UOC May 2013
15. Key words and phrases visualized in the essay context. Sentences in
light-grey (green) background are key sentences as extracted by the
EssayAnalyser (the number at the start of the sentence indicates its
key-ness ranking); bigrams are indicated in bold (red) and boxed.
DMW UOC May 2013
16. The structural elements of the essay can be used jointly with
the key word extraction to highlight relevant information within
specific parts of the essay, here the introduction (and the
assignment question)
DMW UOC May 2013
17. Key words and phrases as separate lists
DMW UOC May 2013
18. Dispersion of key words across the essay
http://www.open.ac.uk/iet/main/research-scholarship/research-projec
DMW UOC May 2013
19. Can we find ways of using graph visualization
techniques on the key words and key sentences, to
make them helpful and meaningful to students?
DMW UOC May 2013
20. Final Thoughts
• Instant machine
feedback not prevalent
• Artificial Intelligence
analysis to tutors
leading to Wizard of Oz
responses (Shaffer,
2013)
• Just in time feedback is
the ultimate goal
DMW UOC May 2013
21. References (1)
Arnold, K.E. (2010). Signals: applying academic analytics, Educause
Quarterly, 33(1), p10.
http://www.educause.edu/ero/article/signals-applying-academic-analytics
(Accessed 30 April 2013)
Bastien, M., Heymann, S. & Jacomy, M. (2009). Gephi: an open source
software for exploring and manipulating networks. Paper presented at
the International AAAI Conference on Weblogs and Social Media.
Bull, S & Kay, J. (2007). Student models that invite the learner in: the
SMILI:-) open learner modelling framework, International Journal of
Artificial Intelligence in Education, 17(2).
Campbell, J.P. & Oblinger, D.G. (2007). Academic Analytics,
Educause.
Dawson, S., Bakharia, A. & Heathcote, E. (2010). Snapp: Realising the
affordances of real-time SNA within networked learning environments.
Paper presented at The 7th
International Conference on Networked
Learning, Aalborg, Denmark (3-4 May).DMW UOC May 2013