Workshop presentation using the Potter Box model of ethical reasoning to discuss concerns and dilemmas of Learning analytics - Open Discovery Space and Learning Analytics Community Exchange projects #laceproject #ods_eu

1. Ethical questions
and dilemmas of
Learning Analytics
Tore Hoel, Henri Pirkkalainen, Kati Clements,
Thomas Richter, Thomas Kretschmer
and Christian M. Stracke1
eConference, Belgrade, September 2014
Co-organised with
laceproject.eu

2. Outline
• What is Learning Analytics
• Scenarios of the quantified learner
• Workshop to gather (ethically reflected) solutions
• Introduction to Potter Box – a method used for the
developing the solutions
• Ethical questions, dilemmas and solutions
2

3. What is Learning Analytics?
“actionable insights through problem
definition and the application of statistical
models and analysis against existing and/or
simulated future data”
Cooper, A. 2012 – Cetis Analytics Series What-is-Analytics-Vol1-No-5
3

4. From Data to Insights
Data

5. From Data to Insights
Data Analytics

6. From Data to Insights
Data Analytics Insight

7. From Data to Insights
Data Analytics Insight
Who?
Institution
Tutor
Self
…
Educational
Commercial

8. From Data to Insights
Data Analytics Insight
Who?
Institution
Tutor
Self
…
Educational
Commercial
How?
Social network
Discourse
Content
Disposition
Context
…
Administration

9. From Data to Insights
Data Analytics Insight
Who?
Institution
Tutor
Self
…
Educational
Commercial
How?
Social network
Discourse
Content
Disposition
Context
…
Administration
What?
Platform
Service
…
Availability
Access

10. From Data to Insights
Data Analytics Insight
Who?
Institution
Tutor
Self
…
Educational
Commercial
How?
Social network
Discourse
Content
Disposition
Context
…
Administration
What?
Platform
Service
…
Availability
Access

11. From Data to Insights
Why?

12. Handling ethical dilemmas
12
• Finding the signal in the noise,
patterns in the chaos (Silver, 2012)
• “Data and data sets are not objective;
they are creations of human design.
We give numbers their voice, draw
inferences from them, and define
their meaning through our
interpretations” (Crawford, 2013)

13. Data Flows…
… watch
Contexts
Integrity Norms

23. ... and now, – the
workshop
Using a ethical approach, following the 4 steps of
the Potter Box
27

24. The task
We want some advice!
You should give me some ethical & valid solutions:
We have concerns about Privacy in LA.
What is the most serious concern?
What is your recommendation / solution (that stand an
ethical test)?
E.g., Concern: Control of data. Solution: Learner should control all
use of their own data.
28

25. The Potter Box Model of
Reasoning
Source: myweb.arbor.edu/rwoods/Media_Ethics7/intro.htm.ppt

26. The “Potter Box”
Source: myweb.arbor.edu/rwoods/Media_Ethics7/intro.htm.ppt
• Dr. Ralph Benajah
Potter, Jr.
• Professor of Social
Ethics Emeritus
BA, Occidental
College
BD, McCormick
Theological Seminary
ThD, Harvard
University
Ralph Benajah Potter, Jr., who retired in
July 2003, began teaching at HDS in
1965. He is an ordained Presbyterian
minister and the author of the book War and
Moral Discourse and assorted scholarly
articles. He is a founding fellow of the
Hastings Center for Bioethics and is a
member of the American Academy of
Religion, the Society for Christian Ethics,
Societe Europeene de Culture, the Society for
Values in Higher Education, and, at Harvard,
the Senior Common Room of Lowell House.
His 1997 HDS Convocation Address was
titled "Moralists, Maxims and Formation for
Ministry."
Source:http://www.hds.harvard.edu/faculty/em/potter.html

27. Four Dimensions of Moral Analysis
Definition -
Establishing facts
↓ ↑
Values -
Justification
Source: myweb.arbor.edu/rwoods/Media_Ethics7/intro.htm.ppt
Loyalties
→ Principles

28. Use of Ethical Principles
No conclusion can be morally justified without a clear
demonstration that an ethical principle shaped the final decision.
What Actually Happens What Ought to Happen
Definition
Problem
Values Principles
Source: myweb.arbor.edu/rwoods/Media_Ethics7/intro.htm.ppt
Loyalties
Descriptive Normative

29. Potter Box applied to a school case I
SITUATION
Teacher introduces an app that
leaks data to 3rd party
VALUES / JUSTIFICATION
Students are motivated and learn
better
Teacher trust 3rd party company
JUDGEMENT
The school has to inform better about
digital learning practices and support
transparency
LOYALTIES
To the learners. To app provider
To the teacher and the results
PRINCIPLE
Respect for Individual Integrity
Accountability of industry

30. Potter Box' 4 steps
Empirical Definition
Identifying Values
Source: myweb.arbor.edu/rwoods/Media_Ethics7/intro.htm.ppt
Particular Judgement
or Policy
Choosing Loyalties
Appeal to Ethical
Principles
Sociological
Immediate
External
Philosophical
Reflective
Internal
both positive
& negative
Feedback
virtue, duty, utility,
rights, love
Facts

31. Ethical questions & dilemmas
• Does the administration let students know their
academic behaviors are being tracked?
• What and how much information should be provided to
the student?
• How much information does the institution give the
teachers?
• Does the institution provide a calculated probability of
success or just a classification of success (e.g., above
average, average, below average)?
35

32. Ethical questions & dilemmas
• How should teachers react to the data? Should the
teacher contact the student? Will the data influence
perceptions of the student and the grading of
assignments?
• What amount of resources should the institution invest in
students who are unlikely to succeed in a course?
• What obligation does the student have to seek
assistance?
36
From: Willis, J. E., III, Campbell, J., & Pistilli, M. (2013). Ethics, big data, and analytics: A model for application.

33. More questions
• What are the dangers in learning analytics?
• Is “raw data” an oxymoron?
• Should students be allowed to opt-out of having their
personal digital footprints harvested and analysed?
• To what extent should students have access to the
content of their digital dossiers, who have access to these
dossiers, and what it is used for?
• How complete and permanent a picture do our data
provide about students?
37

34. More questions
• To what extent do we provide students the option to
update their digital dossiers and provide extra (possibly
qualitative) data?
• Do students have the right to request that their digital
dossiers be deleted on graduation?
• If we outsource the collection (and analysis) of student
digital data to companies, do students need to give
consent? [Who owns a student’s data?]
• Is bigger data sets always better or provide more
complete pictures?
• What responsibility comes with ‘knowing’?
38

36. “Ethical questions and dilemmas of Learning Analytics ” workshop facilitated by Tore Hoel, Oslo and
Akershus University College of Applied Sciences, was held at eConferece, Belgrade, 23 September 2014.
Presentation in co-operation with Henri Pirkkalainen & Kati Clements (University of Jyväskylä), and
Thomas Richter, Thomas Kretschmer & Christian M. Stracke (University of Duisburg-Essen).
For further information: tore.hoel@hioa.no
@tore
This work was undertaken as part of the LACE Project and Open Discovery Space project, both projects
supported by the European Commission
These slides are provided under the Creative Commons Attribution Licence:
http://creativecommons.org/licenses/by/4.0/. Some images used may have different licence terms.
40
www.laceproject.eu
@laceproject
opendiscoveryspace.eu