1. Learning Analytics@UC3M
Carlos Delgado Kloos, Abelardo Pardo (U. Sydney),
Pedro J. Muñoz-Merino, Israel Gutiérrez, Derick Leony
Universidad Carlos III de Madrid
www.it.uc3m.es/cdk
www.emadridnet.org
2. Education?
Just knowledge transfer?
One way process?
Teaching
K K’>K
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
3. Feedback!
Feedback is needed to
Control that learning occurs
Take adequate measures if not
K K’
K’>K?
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
4. Assessment
Simplest form of feedback
Formative: to support learning
Summative: also grading function
Teaching
Assessment
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
5. Analytics
Another form of feedback
Recollection of educational information
Can be done especially well in the digital world
Teaching
Analytics
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
7. 1. Capturing Events
CCOLAB system
Capture raw events generated
Use virtual machine
Capture
of data
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
10. Events captured
Power up and shutdown of virtual machine
Commands used in the command line interface (bash)
Execution of the compiler and errors obtained (gcc)
Execution of the debugger and commands used (gdb)
Execution of the editor and IDE (kate & kdevelop resp.)
Execution and outcome of
the memory profiler (valgrind)
Pages visited with the browser
(Firefox)
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
12. References
A. Pardo, C. Delgado Kloos:
“CCOLAB UC3M: Events recorded in an undergraduate
collaborative C Programming Course”.
In: DataTel 2010
V.A. Romero, A. Pardo, D. Burgos, C. Delgado Kloos:
“Monitoring Student Progress Using Virtual Appliances:
A Case Study”.
Computers & Education 58: 4, May 2012, 1058–1067
Lead researcher: Abelardo Pardo
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
13. 2. Visualization
LearnGlass: Platform to facilitate the creation
of visualizations of learning events
Implements generic requirements:
User and permission management
Data filters
Visualization dashboard
Visualization
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
14. 14
LearnGlass Dashboard
3 variations
of the same
visualization
cdk@it.uc3m.es
Berlin, Germany EDUCON, Berlin, Germany, 2013-03-12--15
EDUCON, 2013-03-12--15
15. 15
Modular Architecture
LearnGlass can be extended
through the installation of modules
Each module provides:
Visualization interface
An optional light visualization for the dashboard
There are currently two modules available:
Activity report
Learners with most and least events
cdk@it.uc3m.es
Berlin, Germany EDUCON, Berlin, Germany, 2013-03-12--15
EDUCON, 2013-03-12--15
16. 16
View of a module interface
cdk@it.uc3m.es
Berlin, Germany EDUCON, Berlin, Germany, 2013-03-12--15
EDUCON, 2013-03-12--15
17. References
D. Leony, A. Pardo, L. de la Fuente Valentín,
D. Sánchez de Castro, C. Delgado Kloos:
“GLASS: A Learning Analytics Visualization Tool”.
In: International Conference on Learning Analytics and
Knowledge. ACM New York, USA, 2012, 162-163
Lead researchers: Abelardo Pardo, Derick Leony
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
18. 3. Abstracting Out Data
Genghis project at UC3M
Khan Academy platform
deployed locally Aug 2012
Flipping the classroom in
remedial course in Physics
Analyse online student behaviour
online
Visualization Capture
of data
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
19. Low-level analytics provided
Progress summary
Daily activity report
Skill progress over time
Activity by day
Badges earned
Activity on exercises
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
20. Higher-level analytics inferred
Time distribution
Correct progress on the platform
Efficiency in learning
Gamification habits (influence of badges)
Exercise solving habits (hint abuse, hint avoidance, …)
Activity habits (explorer or recommendation follower)
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
22. References
P.J. Muñoz-Merino, J.A. Ruipérez, C. Delgado Kloos:
“Inferring Higher Level Learning Information from
Low Level Data for the Khan Academy Platform”.
In: LAK2013: 3rd Conf. Learning Analytics and
Knowledge, 8-12 Apr. 2013, Leuven (Belgium)
Lead researcher: Pedro Muñoz-Merino
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
23. 4. Just-in-Time Teaching
ClassON system (in-CLass Analytics
for aSSessment and OrchestratioN)
Use of learning analytics in face-to-face sessions
awareness of students activity
information to orchestrate in a better way
face-to-face
Visualization Capture
of data
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
24. Student component
Web-based
Integrated in the problem assignment
Monitor student interactions
progression
questions
timing
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
25. Teacher component
Web app for tablets or smartphones
Information attached to the physical space
Teacher is aware of student’s
location
name and photo
Progression
questions
waiting times
And (s)he’s more informed to make decisions!
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
27. References
I. Gutiérrez Rojas, R.M. Crespo, C. Delgado Kloos:
“Orchestration and feedback in lab sessions:
improvements in quick feedback provision”.
In: EC-TEL 2011: 6th European Conf. of Technology
Enhanced Learning, 20-23 Sep. 2011, Palermo (Italy),
Proc. LNCS 6964. Springer 2011
Lead researcher: Israel Gutiérrez
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
28. 5. Gamification
ISCARE system
Analytics for motivation and joy: Adaptation of the Swiss-
system of competition (used eg. in chess) to education
Motivating and fair system:
Different rounds. No elimination
For each round, pairings of 2 people that compete
Assign each person an opponent with close scoring
Active learning: solving exercises
Evaluation
Interoperability: IMS QTI
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
30. Analytics in ISCARE
Evolution of scores for each student in each round
The difficulty of the opponents
the student competed against
Exercises assigned to each student and
if (s)he answered them correctly or not
Information about each exercise:
number of times it was presented or solved correctly
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
32. References
M. Fernández Molina, P.J. Muñoz-Merino,
M. Muñoz-Organero, C. Delgado Kloos:
“Educational Justifications for the Design of the ISCARE
Computer Based Competition Assessment Tool”.
In: Internat. Conf. Web Based Learning, 2011, 289-294
P.J. Muñoz-Merino, M. Fernández Molina,
M. Muñoz-Organero, C. Delgado Kloos:
“An Adaptive and Innovative Question-driven Competition-
based Intelligent Tutoring System for Learning”.
Expert Systems with Applications, 39(8), 2012, 6932-6948
Lead researcher: Pedro Muñoz-Merino
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
33. Conclusion
In all important processes, the behaviour is monitored
(Quality control)
Assessment is not enough
Prerequisite for scaling: MOOCs
Definitely helps in improving education
Privacy concerns have to be taken care of
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15