How can learning analytics be taken from its design to its deployment in an educational institution? What are the issues, limitations, strategies? This presentation includes a descirption of Learning Analytics, examples, how to tackle systemic deployment and suggestions on how to build institutional capacity.
Learning and Behavioral Analytics From concept to reality
1. Will Scullin Flickr
Learning and Behavioral Analytics
From concept to reality
Dr Abelardo Pardo (@abelardopardo)
Associate Head of Learning and Teaching
School of Electrical and Information Engineering
slideshare.net/abelardo_pardo
Taylor’s University
Academic Leaders Retreat
4 March 2014
2. jbgeronimi flickr.com
“How to measure the effectiveness of fye activities
and ensure we r on d right track?” Yesterday by CT
“No data, no talk” Yesterday by DVC Pradeep Nair
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
2
3. Unhindered by Talent flickr.com
Many colleges and universities have demonstrated
that analytics can help significantly advance an
institution in such strategic areas as resource
allocation, student success, and finance
(Bichsel, 2012)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
3
4. cygnus921 Flickr
What is LA?
Current
Projects
Systemic
Deployment
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
Building
Institutional
Capacity
4
5. cygnus921 Flickr
What is LA?
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
5
6. NYTimes 16/02/12
Is she pregnant?
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
6
7. Daily Mail UK 20/03/12
TV is watching you
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
7
8. WSJ 19/07/12
How do you read?
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
8
10. Humans are Subjective
Mind Sights: Original Visual Illusions, Ambiguities, and Other Anomalies, With a Commentary on the Play of Mind in Perception and Art, Roger N. Shepard, 1990
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
10
11. Learning Analytics
Measurement, collection, analysis
and reporting of data about learners
and their contexts, for purposes of
understanding and optimizing
learning and the environments in which
it occurs.
(US Dept. of Education)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
11
12. Kurafire flickr.com
Feedback as a dialogue
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
12
13. Discipline in its early steps,
but with large potential.
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
13
14. (Manyika et al., 2013)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
14
17. Werner Kunz Flickr.com
Predict
Refine
Act
The five
steps of
analytics
Report
Collect
(Campbell, De Blois, Oblinger 2007, Academic Analytics, EDUCAUSE)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
17
18. Thomas Leth-Olsen flickr.com
Knowledge
Discovery in
Databases
Technology
enhanced
learning
Social
network
analysis
Business
Intelligence
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
Educational
modelling
18
19. zzpza flickr.com
• Statistical prediction
• Clustering and profiling
• Relationship mining
• Link prediction in
networks
• Text analysis
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
19
21. measure Joe flickr.com
1
2
3
4
Abelardo Pardo
Context: Student
retention
Questions: Why do
they leave? Can it be
avoided?
Data + algorithms: ?
Level of intervention: ?
Learning and Behavioral Analytics. From concept to reality
21
22. Shaylor flickr.com
1
2
3
4
Context: Student success
Questions: How did they
learn?
Data + algorithms: ?
Level of intervention: ?
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
22
23. cygnus921 Flickr
What is LA?
Abelardo Pardo
Current
Projects
Learning and Behavioral Analytics. From concept to reality
23
24. Joe Fruchey flickr.com
MYTH: It is a highly sophisticated technique
It can be deployed gradually
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
24
25. tudedude flickr.com
MYTH: LA is a tool to fix a problem.
LA is a process to build/refine a model
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
25
26. HikingArtist flickr.com
MYTH: Once we have a
model, we apply it to all
scenarios
Models are derived from
the scenarios
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
26
27. DullHunk flickr.com
Reduce attrition. Phone 10 students this semester.
Which students? When?
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
27
28. colecamp flickr.com
A/B Testing: Space usage after change in
course/curriculum
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
28
29. (Macfadyen & Dawson, 2010)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
29
30. (Romero, Ventura & García, 2008)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
30
32. Signals
Nottsexminer
• Run by the instructor
• Points so far (Gradebook)
• Time on task (LMS)
• Past performance (SIS)
• Instructor chooses
intervention
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
32
33. Rio Salado (AZ, USA)
• Enrolment figures
• LMS variables
• Thirty factors considered
• Separate course models
• Naïve Bayes classification
method
• Model is applied to classify
students
http://campustechnology.com/Articles/2011/12/14/Monitoring-the-PACE-of-Student-Learning-Analytics-at-Rio-Salado-College.aspx?p=1
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
33
34. www.snappvis.org (Last accessed Oct. 2013)
Bookmarklet in your browser
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
34
35. Learning Glass
(Leony et al., 2012)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
35
36. Data uses
• Multiple levels of intervention
• Variety of data sources
• Commonalities vs differences
• Require multidisciplinary
work
(Bichsel, 2012)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
36
41. puthoOr Photography flickr.com
LMS interactions, quizzes, response times, sessions,
hints requested, questionnaires, interviews, requests,
demographics, high school grades, etc.
(Bienkowski, M., Feng, M., & Means, B., 2012)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
41
47. (Powell & MacNeil, 2012)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
47
Salfalko flickr.com
Data Wrangler: someone comfortable handling
statistics, manipulating data for visualisation, and
capable of engaging with academics about the student
experience and course design. It is their job to
experiment with different tools to interpret, visualise
and share information with academics as a basis for
gaining actionable insights.
49. cygnus921 Flickr
What is LA?
Current
Projects
Systemic
Deployment
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
Building
Institutional
Capacity
49
50. Impact Levels
gregoriosz
• Micro: at risk students,
insight to learners,
recommendations
• Meso: clear understanding of
objectives, increase
productivity, help leaders
• Macro: institutional
transformation
(Buckingham Shum, 2012)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
50
51. Phase 1
Phase 2
Derpunk flickr.com
Static Reports
Dynamic
Analysis and
Intervention
Optimization
Phase 3
(Norris & Baer, 2013)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
51
52. Allie Caufield flickr.com
Student Success and Retention
Practices
Workflows
Processes
Policies
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
52
53. juhansonin flickr.com
1) Develop culture of using data to make decisions
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
53
54. Rosellsgirl flickr.com
2) IT dept. are key to initiate connection
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
54
55. Not so much flickr.com
3) Multidimensional from the start
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
55
56. Abraham C flickr.com
4) Gradual deployment
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
56
57. Fhemerick flickr.com
5) Interfaces that provide value to stakeholders
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
57
58. Doug Woods flickr.com
6) Refinement as intrinsic component of the process
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
58
59. Laverrue flickr.com
7) Privacy and Ethics
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
59
60. Pennstatenews flickr.com
8) Keep the student at the center
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
60
61. Will Scullin Flickr
Learning and Behavioral Analytics
From concept to reality
Dr Abelardo Pardo (@abelardopardo)
Associate Head of Learning and Teaching
School of Electrical and Information Engineering
slideshare.net/abelardo_pardo
Taylor’s University
Academic Leaders Retreat
4 March 2014
62. References
Arnold, K. E. (2010).
Signals: Applying Academic Analytics.
EDUCAUSE Quarterly, 33(1), 10.
Bichsel, J. (2012).
Analytics in Higher Education Benefits, Barriers, Progress, and Recommendations.
EDUCAUSE Center for Applied Research
Bienkowski, M., Feng, M., & Means, B. (2012).
Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics. US
Department of Education.
Buckingham Shum, S. (2012).
Learning analytics. UNESCO policy brief.
Retrieved from http://iite.unesco.org/pics/publications/en/files/3214711.pdfhttp://
iite.unesco.org/pics/publications/en/files/3214711.pdf, March 2014
Buckingham Shum, S. (2013).
Building Analytics Capability @open.edu
Retrieved from
http://www.slideshare.net/sbs/building-analytics-capacity-openeduhttp://www.slideshare.net/sbs/buildinganalytics-capacity-openedu, March
2014
Campbell, J. P., DeBlois, P. B., & Oblinger, D. G. (2007).
Academic Analytics.
Educause Review (Vol. 42). EDUCAUSE White Paper.
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
62
63. References 2
Dawson, S. (2010).
“Seeing” the learning community: An exploration of the development of a resource for monitoring online
student networking.
British Journal of Educational Technology, 41(5), 736–752.
Ferguson, R. (2012).
The State of Learning Analytics in 2012: A Review and Future Challenges a review and future challenges.
Technical Report Knowledge Media Institute KMI–12–01). The Open University UK.
Greller, W., & Drachsler, H. (2012).
Translating learning into numbers: A generic framework for learning analytics.
Educational Technology & Society, 15(3), 42–57.
Macfadyen, L. P., & Dawson, S. (2010).
Mining LMS data to develop an “early warning system” for educators: A proof of concept.
Computers & Education, 54(2), 588–599
Manyika, J., Chui, M., Groves, P., Farrell, D., Kuiken, S. Van, & Doshi, E. A. (2013).
Open data: Unlocking innovation and performance with liquid information
McKinsey Global Institute (p. 116).
Norris, D. M., & Baer, L. L. (2013).
Building Organizational Capacity for Analytics.
Educause.
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
63
64. References and 3
Powell, S., & MacNeil, S. (2012).
Analytics Series Institutional Readiness for Analytics (Vol. 1, pp. 1–11).
JISC Center for Educational Technology & Interoperability Standards, Analytics Series, 1(8).
Romero, C., Ventura, S., & Garcia, E. (2008).
Data mining in course management systems: Moodle case study and tutorial.
Computers & Education, 51(1), 368–384.
Siemens, G., & Long, P. (2011).
Penetrating the Fog: Analytics in Learning and Education.
Educause Review, 48(5),
Siemens, G. (2013).
Learning Analytics: The Emergence of a Discipline.
American Behavioral Scientist, 57(10), 1380–1400.
Siemens, G., Dawson, S., & Lynch, G. (2013).
Improving the Quality and Productivity of the Higher Education Sector. Policy and Strategy for
Systems-Level Deployment of Learning Analytics.
Technical Report, Office for Learning and Teaching, Australian Government
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
64