Mary Loftus #ILTAEdTech - Ways of Seeing Learning - 2017 v0.6

1. • Ways of Seeing Learning
Learning Analytics for Learners
Mary Loftus Michael Madden
NUI Galway NUI Galway
mary.loftus@nuigalway.ie michael.madden@nuigalway.ie
@marloft
• The authors acknowledge the support of Ireland’s Higher Education Authority through the IT Investment Fund and ComputerDISC in NUI
Galway.
1

2. 2

3. And today…
• Data, Artificial Intelligence &
Machine Learning are having a
similar effect across society
• This revolution is not only
showing us to ourselves in new
ways – it is shaping how we live.
3

4. Learning Analytics – a Definition
• “Learning analytics is the measurement, collection, analysis and
reporting of data about learners and their contexts, for purposes of
understanding and optimising learning and the environments in
which it occurs”
• Call for Papers of the 1st International Conference on Learning Analytics & Knowledge (LAK 2011)
Learning Sciences
Data Mining
Data Visualization
Psychology 4

5. Central to education’s purpose is “the coming into presence of unique individual beings”
Education “spaces might open up for uniqueness to come into the world”
– Biesta, G. J. J. (2015). Good Education in an Age of Measurement: Ethics, Politics, Democracy. Routledge.
5

6. Learning Analytics – The Story So Far
• Predicting student outcomes - identifying ‘at-risk students’
• Personalisation of student learning
• Multi-modal analytics – analyses of audio, video, location data
• Discourse and writing analytics
• Measuring ‘student engagement’ & disengagement
• Levels of Learning Analytics:
• Teachers
• Course
• Institution
• National
But little work done from
the Student perspective
6

7. If we can make learning more visible,
can we...
• Reduce the need for formal testing and
examinations?
• Do more problem-based learning & assessment?
• Provide more formative feedback for students?
• Model student’s conceptual understanding?
• Support metacognition?

8. Student Vulnerability, Agency, and
Learning Analytics
• Prinsloo & Slade examine how we:
• decrease student vulnerability,
• increase student agency,
• empower students as participants in learning analytics
• moving students from quantified data objects to qualified and
qualifying selves
• “In light of increasing concerns about surveillance,
higher education institutions (HEIs) cannot afford a
simple paternalistic approach to student data”
• Prinsloo & Slade (2016)
8

9. Measuring Gets Results – But Care
Needs to be Taken…
• When we measure and intervene
accordingly, we can clearly see
improvement and impact
• However, the Hawthorne effect is a
kind of "tell-me-what-you-measure-I-
will-tell-you-how-people-react-to-it"
effect
9

10. Data is Political
• We need to take care in our research to ensure fairness and not replicate
societal biases and discrimination
• There is nothing about doing data analysis that is neutral.
What and how data is collected, how the data is cleaned and
stored, what models are constructed, and what questions are
asked—all of this is political.
dana boyd (2017)

11. Unintended Consequences…
• Criminal Justice Systems – discriminating on the basis of race?
• Employment Screening – address, age, gender?
• Advertising – different ads served depending on race? (Sweeney 2013)
• Even if humans are there as a ‘final check’, there is potential for ‘Moral
Crumple Zones (Elish 2016)
Most data analysis makes prejudicial decisions as part of
clustering without having any understanding of the people or
properties that they are using. It’s merely math! But that math—
and the decisions that are determined by it—have serious social
ramifications.
dana boyd (2017)

12. White-Box Algorithms & Transparency
• Students (and citizens) need to be able to ‘see into’ algorithms that impact
their opportunities and quality of life.
The problem with contemporary data analytics is that we’re
often categorizing people without providing human readable
descriptors.
dana boyd (2017)
12

13. “Action can never manifest through a predictable, deterministic series
of consequences, since the subject, by acting, is placed within a
complicated web of relationships which cannot be predicted before
hand. In the same sense, Action is irreversible.”
Hannah Arendt
“For apart from inquiry, apart from the praxis, individuals cannot be
truly human.
Knowledge emerges only through invention and re-invention, through
the restless, impatient, continuing, hopeful inquiry human beings
pursue in the world, with the world, and with each other.”
Paulo Freire
13

14. Research Values & Ethical Grounding
• Grounded in the Student perspective
• Students as owners of their learning data
• Links learning analytics to learning design
• Machine Learning with an emphasis on white-box
modelling & visibility as well as prediction
• Data literacy capacity building for students
• An application to the NUIG Research Ethics Committee
was approved in Feb 2017
14

15. Research Questions
1. Can a Learning Analytics system provide an interface for
students to engage in metacognitive activities around
their own learning, thereby improving individual
learning?
2. Can we retool an existing learning analytics system using
machine learning, modelling and classifiers to provide
this metacognitive interface to students?
3. Can such a system help students visualize, track and
reflect on their own learning and development goals
and help them to improve performance? 15

16. 4. Metacognition
1. Student
Actions
2. Bayesian
Models & Open
Learning Models
3. Student
Reflection
16

17. Example of Simple Bayesian Network
Madden et al (2008)
17

18. Open
Learner
Models
Bull, Kay et al (2016)
18

19. Data Gathering
• Data Gathered so far
• Internet of Things module - 1st Yr
Computing Students activity in Moodle
(Blackboard equivalent)
• Student activity in Github & Trello Connected Learning
Analytics Toolit
Kitto et al (2016)
19

20. Methodology
• Qualitative Research – interviews with students, showing them some
of their data and finding out what data interests them
• Identifying suitable Data Models for showing students their own data
• Next Stages:
• Present models to Students
• Identify Machine Learning & Modelling Enhancements to improve understanding
• Refine models further
• Reflect to students again
• Identify positive effects on student learning experience
20

21. Research Timeline
• Literature
Review
• Research
Questions
Ethical
Approval
• Data Modelling
• Qualitative
Research
Data
Gathering • Share improved
models with
students
• Assess impact
Write up
2017 2018 2019
21

22. Presentation References
• Biesta, G. J. J. (2015). Good Education in an Age of Measurement: Ethics, Politics, Democracy. Routledge.
• boyd, danah. (2017, April 12). Toward Accountability. Retrieved 18 April 2017, from
https://points.datasociety.net/toward-accountability-6096e38878f0
• Bull, S., Ginon, B., Boscolo, C., & Johnson, M. (2016). Introduction of learning visualisations and metacognitive
support in a persuadable open learner model. In Proceedings of the Sixth International Conference on
Learning Analytics & Knowledge (pp. 30–39). ACM. Retrieved from http://dl.acm.org/citation.cfm?id=2883853
• Elish, M. C. (2016). Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction (We Robot 2016)
(SSRN Scholarly Paper No. ID 2757236). Rochester, NY: Social Science Research Network. Retrieved from
https://papers.ssrn.com/abstract=2757236
• Millán, E., Loboda, T., & Pérez-de-la-Cruz, J. L. (2010). Bayesian networks for student model engineering.
Computers & Education, 55(4), 1663–1683. https://doi.org/10.1016/j.compedu.2010.07.010
• Sweeney, L. (2013). Discrimination in Online Ad Delivery. Queue, 11(3), 10:10–10:29.
https://doi.org/10.1145/2460276.2460278
• Madden, Michael G. (NUI, Galway), Lyons, William and Kavanagh, Ita (Limerick Institute of Technology).“A
Data-Driven Exploration of Factors Affecting Student Performance in a Third-Level Institution”, Proceedings of
AICS-2008: 19th Irish Conference on Artificial Intelligence and Cognitive Science, Cork, August 2008.
• Other refs in Abstract 22

23. Thank you
Mary Loftus
@marloft
mary.loftus@nuigalway.ie
23