This document reviews learning analytics and identifies challenges in the field. It discusses how learning analytics measures and analyzes data from learners and learning environments to understand and optimize learning. Key drivers include the availability of large datasets from online learning systems and political priorities around improving education outcomes. Challenges involve extracting value from big data, optimizing learning opportunities, and using data to substantially improve education at various levels. The document also outlines the relationships between learning analytics, educational data mining, and academic analytics research areas.
2. Learning analytics
The measurement, collection,
analysis and reporting of data
about learners and their contexts,
for purposes of
understanding and optimising learning
and the environments in which it occurs.
LAK 11 Call for Papers
3. Driver 1: Big data
Formal: Blackboard, Moodle
Informal: Facebook, OpenLearn
•Interaction data
•Personal data
•Systems information
•Academic information
•Social information
7. Driver 3: Political concerns
The goal of creating an interconnected feedback system would be to
ensure that key decisions about learning are informed by data and
that data are aggregated and made accessible at all levels of the
education system for continuous improvement.
US Department of Education, 2010
Table: http://www.oecd.org/document/2/0,3746,en_2649_39263238_48634114_1_1_1_1,00.html
8. Political challenge
How can we substantially improve
learning opportunities and educational
results at national or international levels?
9. Three main interest groups
Learners
Governments
and teachers
Schools and
colleges
10. Data-driven analytics
Use of web mining techniques
•Clustering
•Classification
•Outlier detection
•Association rule mining
•Sequential pattern mining
•Text mining
11. Effective better learners
The goal:
Turn learners into effective better learners
Focus on:
data mining and machine learning techniques…
to enhance web-based learning environments for
the educator to better evaluate the learning process,
as well as for the learners to help them in their
learning endeavour
Zaïane, 2001
12. Learning-focused perspectives
Knowledge is constructed
through social negotiation
Learning takes place in
networks and in
communities of practice
Learning can be scaffolded
by a more able other Dawson, McWilliam, Tan, 2008
13. Strategic perspectives – 2007/8
by 2020 the overall portion of the U.S.
workforce with a college degree will
be lower than it was in 2000
analytics is emerging as a new
tool that can address what seem
like intractable challenges
analytics has the potential to
improve teaching, learning, and
student success
14. EDM and analytics split
• extend geographical focus
• make tools easier for educators to use
• standardize methods and data across systems
• integrate tools within e-learning environments
• develop education-specific mining techniques
16. Dividing responsibility
Educational data mining
How can we extract value from these big sets of
learning-related data?
Academic (and action) analytics
How can we substantially improve learning
opportunities and educational results at national or
international levels?
Learning analytics
How can we optimise opportunities for learning?
17. Meeting the challenge
How can we optimise opportunities for learning?
•Maintain focus on this challenge
•Learn from previous work in all three fields
•Integrate experience from different disciplines
•Focus on learners and teachers
18. Fresh challenges
• Widen range of theory-driven approaches
• Develop methods of presenting analytics clearly
• Adopt standards for the structure and export of data
• Broaden focus from higher education
• Broaden international focus
• Address issues around ethics, privacy and data
• Explore possibilities offered by new data sources
Editor's Notes
I work at the UK’s Open University – my department has been collecting and analysing data about the university’s students for more than 40 years Here today as a member of the SOLAR steering group Aim is that this presentation, and our discussion, will enable us to develop a shared understanding of how the field has emerged and what it it aiming to achieve
Definitions of learning analytics have changed over the last decade. A set of processes focused on learning. Unstated – ‘data’ refers to large, machine-readable datasets that were generated for other purposes.
First the drivers, and their associated challenges 1. The rise of big data in business, moving to education
How can we draw on records that are distributed across different sites with different standards, owners and levels of access?
2. US has 6 million online learners. Benefits and problems. Disorientated, technical problems, motivation Teachers can struggle to interpret the learning and participation of individuals when this is buried beneath hundreds of contributions within discussions that have lasted many weeks. Infographic: http://sloanconsortium.org/sites/default/files/pages/OnlineLearningSurvey-Infographic-1.png
Improve the experience for learners and teachers, to improve both learning and the learning environment.
Organisation for Economic Co-operation and Development (OECD) Measure, demonstrate and improve performance These reports can be used to drive political change. Countries benchmark themselves against similar countries and against the rest of the world Table from http://www.oecd.org/document/2/0,3746,en_2649_39263238_48634114_1_1_1_1,00.html
3. looking beyond the learner and the institution to the country as a whole – looking for ways to improve standards nationally.
Three drivers, three challenges and three interest groups These require work on different scales and at different granularities They therefore affect how researchers conceptualise problems, gather data, report and act on findings.
Brief history In the early 21 st -century we were still investigating how learning took place online, so data driven analytics predominated. These used established web mining techniques. Clustering and classification -novices and experts Association rule mining -common characteristics of students who did well, or dropped out. Sequential pattern mining -routes student took through learning materials, text mining - forum interactions. Image taken from Damez, Dang,Marsala, Bouchon-Meunier, 2005
There is perhaps a tendency to assume that the focus was all on data and algorithms. This is not the case. These ideas are from the most referenced paper in data mining This is some indication of the way that the terminology has shifted over the past decade And of the clear links between data mining and learner analytics
Pedagogy neutral – social constructivist view that knowledge is constructed through social negotiation they weren’t particularly associated with any theories about how learning takes place This began to change as social network analysis was introduced to the toolkit. Learning theorists such as Vygotsky and Wenger increasingly began to inform and drive research
Educause had been increasingly active in this area – trying to disambiguate the terminology A flurry of publications in 2007/8 focused attention on analytics in relation to education, and their strategic importance
At this point the move to three stakeholders and three challenges catalysed a split in the field EDM made firmer steps to being a field in its own right – with the first of its annual conferences (none of the paper titles used the term ‘analytics’) Two reviews of the field came out. ROMERO, C. & VENTURA, S. 2007. (1995-2005) BAKER, R. S. J. D. & YACEF, K. 2009. (2009) While the split may have been inevitable, it was a shame because (a) we lost access to the literature and (b) we lost sight of their vision
The split between academic analytics and learner analytics is still taking place The field still doesn’t seem to be broad enough to support another community and conference So academic analytics and learning analytics remain increasingly uneasy bedfellows
Three fields have emerged – each one addressing one of the challenges posed by the original drivers – big data, shifts in the learning landscape, and political concerns about education
This gives us a clear challenge, but it also explains some of the issues that have been problematic at this conference Explains why we are not always focused on learning It explains why we are reformulating ideas that appeared within the EDM community years ago It explains why the input from learning sciences is not as high as we would like And it explains why our attention wanders from learners and teachers, to the concerns of institutions. - But we have been relatively quick to put our house in order
For the future, we have a series of challenges, that have emerged from this conference and from the literature These could be addressed piecemeal by individual scholars and research teams However, today’s summit gives us the chance to move forward together, in a concerted way, with a shared understanding of where we have come from and what we are aiming to achieve