Talk by Rebeca Ferguson (Open University, UK, and LACE project). The promise of learning analytics is that they will enable us to understand and optimize learning and the environments in which it takes place. The intention is to develop models, algorithms, and processes that can be widely used. In order to do this, we need to move from small-scale research within our disciplines towards large-scale implementation across our institutions. This is a tough challenge, because educational institutions are stable systems, resistant to change. To avoid failure and maximize success, implementation of learning analytics at scale requires careful consideration of the entire ‘TEL technology complex’. This complex includes the different groups of people involved, the educational beliefs and practices of those groups, the technologies they use, and the specific environments within which they operate. Providing reliable and trustworthy analytics is just one part of implementing analytics at scale. It is also important to develop a clear strategic vision, assess institutional culture critically, identify potential barriers to adoption, develop approaches that can overcome these, and put in place appropriate forms of support, training, and community building. In her keynote, Rebecca introduced tools, resources, organisations and case studies that can be used to support the deployment of learning analytics at scale

1. Scaling Up
Learning Analytics
Rebecca Ferguson, The Open University, UK

2. The Open University (OU)
• The Open University: largest in UK
• Distance university
• Making use of big data for 45 years
• Informal learning: iTunes, YouTube…
• MOOCs on FutureLearn,
OpenLearn and elsewhere
• Learning analytics research and events
• LACE project
2
open.ac.uk
http://www.laceproject.eu

3. Learning analytics
3
solaresearch.org
…the measurement, collection, analysis and
reporting of data about learners and their contexts,
for purposes of understanding and optimizing
learning and the environments in which it occurs.

4. Educators use analytics to…
• Monitor the learning process
• Explore student data
• Identify problems
• Discover patterns
• Find early indicators for success
• Find early indicators for poor marks or drop-out
• Assess usefulness of learning materials
• Increase awareness, reflect and self reflect
• Increase understanding of learning environments
• Intervene, supervise, advise and assist
• Improve teaching, resources and the environment
4
Dyckhoff, A. L., Lukarov, V., Muslim, A., Chatti, M. A., & Schroeder, U. (2013).
Supporting Action Research with Learning Analytics. Paper presented at LAK13.

5. Learners use analytics to…
• Monitor their own activities and interactions
• Monitor the learning process
• Compare their activity with that of others
• Increase awareness, reflect and self reflect
• Improve discussion participation
• Improve learning behaviour
• Improve performance
• Become better learners
• Learn!
5
Dyckhoff, A. L., Lukarov, V., Muslim, A., Chatti, M. A., & Schroeder, U. (2013).
Supporting Action Research with Learning Analytics. Paper presented at LAK13.

6. Analytics at scale: UK schools
6
• Aligned with
clear aims
• Huge and
sustained
effort
• Agreed
proxies for
learning
• Clear and
standardised
visualisation
• Driving
behaviour at
every level BUT
• Stressed, unhappy learners
• Analytics with little value for learners or educators
• Omission of key areas, such as collaboration

7. Analytics at scale: Course Signals
Developed at Purdue University, USA
7
Arnold, K. E., & Pistilli, M. (2012). Course Signals at Purdue: Using Learning Analytics
To Increase Student Success. Paper presented at LAK12, Vancouver, Canada.

8. Developing institutional strengths
The OU is developing its capabilities in 10 key areas
8
The university
needs world class
capability in data
science to
continually mine
the data and build
rapid prototypes of
simple tools, and a
clear pipeline for
the outputs to be
mainstreamed into
operations
We need to ensure we have the right architecture and processes
for collecting the right data and making it accessible for analytics
– we need a ‘big data’ mind-set
Benefits will be realised through
existing business processes
impacting on students directly
and through enhancement of
the student learning experience
– we will develop an ‘analytics
mind-set’ in
these areas
The strategic roadmap
will build these
capabilities prioritised
using the indicators and
drivers of student success

9. Easily accessible OU data
Learning design and analytics at the OU
9

10. Relating design and outcomes
Learning design and analytics at the OU
10

11. Innovaating in technology-enhanced learning (TEL)
The TEL Complex
11
The many elements of
the ‘TEL Complex’ must
all be taken into account
as an innovation is
designed, developed
and embedded
Scanlon, E., Sharples, M., Fenton-O'Creevy, M., Fleck, J., Cooban, C.,
Ferguson, R., Cross, S. & Waterhouse, P. 2013. Beyond Prototypes. London: TEL Programme.

12. Rapid Outcomes Modelling Approach (ROMA)
The ROMA Framework
12
Ferguson, R., Macfadyen, L., Clow, D., Tynan, B., Alexander, S., & Dawson, S.. (2015). Setting learning analytics in
context: overcoming the barriers to large-scale adoption. Journal of Learning Analytics, 1(3), 120-144.
Adapted from: Young, J., & Mendizabal, E. (2009). Helping researchers become policy entrepreneurs: How to
develop engagement strategies for evidence‐based policy‐making. ODI Briefing Papers. London, UK: ODI.
Define (and
redefine)
your policy
objectives

13. What does success look like?
13
Academic analytics can guide future change
Student perspectives
● Overall, I am satisfied with the quality of this module
● Overall, I am satisfied with my study experience
● I would recommend this module to other students
● I was satisfied with the support provided by my tutor on this module
● I enjoyed studying this module
● This module met my expectations
Academic perspectives
● The students were well prepared
● The students met specified learning outcomes
● The students defined and achieved their own learning goals
University perspectives
● The module enhanced the university’s reputation
● The module aligned well with others
● The module generated income

14. What does success look like?
● Students demonstrate the skills necessary to network, collaborate,
browse and reflect
● Students show progress towards defined learning outcomes
● Students communicate well… when asked to collaborate
● Students access and share links… when encouraged to browse
● Students return to materials... when encouraged to reflect
● Students engage with course content
● Students seek out new challenges
● Students persist when the work is challenging
● Students persist in the face of failure
● Students ask for help… when they are stuck
after several attempts
● Students compare their learning strategies with those of experts
● Students adapt their learning strategies to resemble those of experts
14
Learning analytics help to identify appropriate interventions

15. Policy objectives
OU Strategic Analytics Investment Programme
15
Vision
To use and apply information
strategically to retain students and
enable them to progress and
achieve their study goals.
This vision requires
• Discursive changes
to the communication of data
and analytics
• Procedural changes
in how learners are supported
• Behavioural changes
associated with sustainable
change in learner support.
Define (and
redefine)
your policy
objectives

16. Political context
Mapping people and processes
16
Tynan, B. & Buckingham Shum, S. (2013). Designing systemic learning analytics at the Open University.
http://www.slideshare.net/sbs/designing‐systemic‐learning‐analytics‐at‐the‐open‐university

17. Key stakeholders
OU Strategic Analytics Investment Programme
17
Define
(and
redefine)
your policy
objectives
A community of stakeholders
working in different areas:
• Intervention and Evaluation
• Data Usability
• Ethics Framework
• Predictive Modelling
• Learning Experience Data
• Professional Data
• Student Tools
Key stakeholders are
• University administrators
• Students
• Educators

18. Desired behaviour changes
OU Strategic Analytics Investment Programme
18
Define
(and
redefine)
your policy
objectives
Vision
To use and apply information
strategically to retain students and
enable them to progress and
achieve their study goals.
Desired behaviour changes
• Staff will use and apply
information strategically
• Students will extend their
learning journeys
• Students will complete their
learning journeys
• Students will set learning goals
• Students will work effectively
towards study goals

19. Engagement strategy
OU Strategic Analytics Investment Programme
19
Define
(and
redefine)
your policy
objectives
• Data in action is provided to
stakeholders through a live portal,
enabling them to understand learner
behaviour and make adjustments
and interventions that will have an
immediate positive impact.
• Data on action is a more reflective
process that takes place after an
adjustment or intervention.
• Data for action takes advantage of
predictive modelling and innovation
in order to isolate particular
variables and make changes based
on a variety of analysis tools.

20. Internal capacity to effect change
OU Strategic Analytics Investment Programme
20
Define
(and
redefine)
your policy
objectives
Includes
• Recruitment
• Capacity building
• Developing an ethical
framework for the use of
learning analytics.

21. Monitoring
OU Strategic Analytics Investment Programme
21
Tynan, B. & Buckingham Shum, S. (2013). Designing systemic learning analytics at the Open University.
http://www.slideshare.net/sbs/designing‐systemic‐learning‐analytics‐at‐the‐open‐university

22. Policy objectives
University of Technology, Sydney, Australia
22
Vision
A university where staff and students understand data and, regardless of
its volume and diversity, can use and reuse it, store and curate it, apply
and develop the analytical tools to interpret it.
Teaching and learning
Ensure that all stakeholders have the capacity to understand and interpret
contemporary data‐rich environments.
Research
Enable researchers to think and act differently when designing their
research methodologies and practices.
Administration
Identify opportunities to obtain, generate, visualize, and
communicate data and analyses that can improve
core business outcomes.
University
Mine existing institutional data to identify areas that can provide
direct evidence or assistance to staff and students.

23. Political context
University of Technology, Sydney, Australia
23
Ferguson, R., Macfadyen, L., Clow, D., Tynan, B., Alexander, S., & Dawson, S.. (2015). Setting learning analytics in
context: overcoming the barriers to large-scale adoption. Journal of Learning Analytics, 1(3), 120-144.
• Project initiated and led by Deputy Vice‐Chancellor and
Vice‐President (Teaching and Learning)
• Pilot projects were completed successfully to secure
ongoing funding.
• Critical to the success of the initial pilot projects was the
existence of an Advanced Analytics Institute with
internationally regarded researchers in big data, data
sciences and analytics sciences.
• This enabled the establishment of a Connected
Intelligence Centre.

24. Key stakeholders
University of Technology, Sydney, Australia
24
Define
(and
redefine)
your policy
objectives
• 190 staff attended a one‐day
‘Data Intensive University
Forum’, thus beginning a
university‐wide conversation.
• Working party included Deputy
Vice‐Chancellors, a senior
member of library staff, and
representatives of all faculties
and administrative areas with
relevant expertise
• Stakeholder buy‐in and ongoing
participation in the project have
been critical to its success.

25. Desired behaviour changes
University of Technology, Sydney, Australia
25
Define
(and
redefine)
your policy
objectives
• Provide information that can be
used to decrease student attrition
• Provide a more detailed
understanding of factors affecting
low pass rates in subjects with very
high failure rates over time
• Provide students with more
information about their own study
and engagement patterns
• Enable a more fine‐grained
understanding of the influences of
a range of possible interventions
on pass rates and completions
• Provide valuable input to learning
futures projects

26. Engagement strategy
University of Technology, Sydney, Australia
26
Define
(and
redefine)
your policy
objectives
• Give attention to institutional culture,
ensuring engagement and buy‐in
from key stakeholders through good
communication and governance
• Invest in pilot projects of significant
concern to the university and
reporting of outcomes
• Invest in infrastructure: tools,
applications, services
• Invest in expertise: recruitment of
critical staff
• Provide leadership and engage
institutional leaders

27. Internal capacity to effect change
University of Technology, Sydney, Australia
27
Define
(and
redefine)
your policy
objectives
• Students and staff must be
sufficiently numerate and equipped
to make use of the analyses that
analytics projects produce.
• A subject has been developed and
trialled with staff
• The course develops students’
ability to engage with complex,
extended arguments underpinned
by numerical data as a key to
participation as informed citizens in
issues of significance to our culture
and society
• The course available as an elective
and will become compulsory

28. Monitoring
University of Technology, Sydney, Australia
28
• UTS has been engaged in a
variety of learning analytics
projects to assess scale and
impact
• For example, the Outreach
Programme rings as many
commencing undergraduate
students as possible. Early
results consistently show a
significant decrease in attrition
in the group of students
contacted.
Define
(and
redefine)
your policy
objectives

29. 29
Slides online at www.slideshare.net/R3beccaF
Rebecca Ferguson @R3beccaF
http://r3beccaf.wordpress.com/
Join the LACE project at the LACE SoLAR Flare on
9 October in Milton Keynes, UK, or online
lanyrd.com/2015/laceflare/