V Jornadas eMadrid sobre “Educación Digital”. Pedro Muñoz Merino, Universidad Carlos III de Madrid: Learning analytics for Massive Open Online Courses. 2015-06-30

1. Learning Analytics for Massive
Open Online Courses
Pedro J. Muñoz-Merino
Twiter: @pedmumeTwiter: @pedmume
email: pedmume@it.uc3m.esemail: pedmume@it.uc3m.es
Universidad Carlos III de Madrid

2. 2
Learning Analytics for MOOCs
● The need of Learning Analytics is increased in
MOOCs
 MOOCs enable rich user interactions with
educational activities
 Big amount of data to extract useful conclusions
 Big amount of users increases the need of self-
reflection and augmented vision for teachers
● Learning analytics support is in an early stage in
MOOCs
 Many learning analytics functionality to be developed

3. 3
Challenges of Learning Analytics for
MOOCs
● Technological issues
 Effective processing of the data
 Compatibility between different sources
● Useful indicators
● Useful visualizations
● Know the relationships among indicators. Know
the causes
● Actuators: Recommenders, adaptive learning,
etc.

4. 4
Inference of indicators
● Pedagogical theories
● Best practices
● Adapted models from other systems
● Indicators that can predict other interesting
indicators

5. 5
Examples of indicators I
Learning
profiles
Effectiveness,
efficiency,
interest
Behaviours
Skills
Emotions
Pedro J. Muñoz-Merino, J.A. Ruipérez Valiente, C. Delgado Kloos,
“Inferring higher level learning information from low level data for
the Khan Academy platform.” Proceedings of the Third
International Conference on Learning Analytics and Knowledge,
pp. 112–116. ACM, New York, (2013)

6. Example II: Inference of optional activities
José A. Ruipérez Valiente, Pedro J. Muñoz Merino, Carlos Delgado‐ ‐
Kloos, Katja Niemann, Maren Scheffel, “Do Optional Activities
Matter in Virtual Learning Environments?”,European Conference
on Technology Enhanced Learning, pp 331-344, (2014)

7. 7
Example III: Inference of more precise
indicators: Effectiveness
Pedro J. Muñoz-Merino, José A. Ruipérez-Valiente, Carlos Alario-
Hoyos, Mar Pérez-Sanagustín, Carlos Delgado Kloos, "Precise
effectiveness strategy for analyzing the effectiveness of students
with educational resources and activities in MOOCs", Computers in
Human Behavior, vol. 47, pp. 108–118 (2015)

8. 8
Example IV: Inference of emotions
Derick Leony, Pedro J.
Muñoz-Merino, José A.
Ruipérez-Valiente, Abelardo
Pardo, David Arellano
Martín-Caro, Carlos Delgado
Kloos, "Detection and
evaluation of emotions in
Massive Open Online
Courses", Journal of
Universal Computer Science
(2015) (In press).

9. 9
Visual analytics for Khan Academy
José A. Ruipérez-Valiente, Pedro J. Muñoz-Merino, Derick Leony,
Carlos Delgado Kloos, “ALAS-KA: A learning analytics extension
for better understanding the learning process in the Khan
Academy platform”, Computers in Human Behavior, vol. 47, pp.
139-148 (2015)

10. Level of relationship
•30.3 % hint avoider
•25.8 % video avoider
•40.9 % unreflective user
•12.1% of hint abuser

11. Analysis of optional activities
•76.81% of students who accessed,
did not use any optional activities
•Difference of use per course and
depending on type of optional
activities
•55 goals (50.9% completed)
•40 votes (26 positive, 13 indifferent,
1 negative)
Type of activity
Percentage of activities accessed
0% 1-33 % 34-66% 67-99% 100%
Regular learning
activities
2.48% 51.55% 23.19% 18.84% 3.93%
Optional activities 76.81% 18.43% 4.14% 0.41% 0.21%

12. Level of relationship: optional
activities vs learning activities
Optional
activities
sig.
(2-tailed)
N = 291
Exercises
accessed:
0.429*
(p=0.000)
Videos
accessed:
0.419*
(p=0.000)
Exercise
abandonment:
-0.259*
(p=0.000)
Video
abandonment:
-0.155*
(p=0.008)
Total time:
0.491*
(p=0.000)
Hint abuse:
0.089
(p=0.131)
Hint
avoider:
0.053
(p=0.370)
Follow
recommendations:
-0.002
(p=0.972)
Unreflective
user:
0.039
(p=0.507)
Video
avoider:
-0.051
(p=0.384)
Proficient
exercises
sig. (2-tailed)
N = 291
Optional
activities:
0.553*
(p=0.000)
Goal:
0.384*
(p=0.000)
Feedback:
0.205*
(p=0.000)
Vote:
0.243*
(p=0.000)
Avatar:
0.415*
(p=0.000)
Display
badges:
0.418*
(p=0.000)

13. Clustering depending on
effectiveness
Grouping types of students

14. 14
Prediction models
Learning gains= 25.489 - 0.604 * pre_test +
6.112 * avg_attempts + 0.017 * total_time +
0.084 * proficient_exercises
José A. Ruipérez-Valiente, Pedro J. Muñoz-Merino, Carlos
Delgado Kloos, “A predictive Model of Learning Gains for a Video
and Exercise Intensive Learning Environment”, Artificial
Intelligence in Education conference, pp. 760-763 (2015)

15. 15
Adaptation rules
Pedro J. Muñoz-Merino, Carlos Delgado Kloos, Mario Muñoz-
Organero, Abelardo Pardo, "A Software Engineering Model for
the Development of Adaptation Rules and its Application in a
Hinting Adaptive E-learning System", Computer Science and
Information Systems, vol. 12, no. 1, pp. 203-231(2015)

16. 16
ANALYSE: LA support for open edX
● General information
 http://www.it.uc3m.es/pedmume/ANALYSE/
● Github
 https://github.com/jruiperezv/ANALYSE
● Demos
 https://www.youtube.com/watch?v=3L5R7BvwlDM&featur
● Authors
 José Antonio Ruipérez Valiente, Pedro Jose Muñoz
Merino, Héctor Javier Pijeira Díaz, Javier Santofimia
Ruiz, Carlos Delgado Kloos

17. 17
MOOC of maths
● MOOC of maths:
 Available at: http://ela.gast.it.uc3m.es
 Topics: Units of measurement, algebra, geometry
 High school education for adults
 Generation of educational materials: CEPA Sierra
Norte de Torrelaguna: Diego Redondo Martínez
 28 videos, 32 exercises
 Configuration, support and personalization of the
MOOC platform at Univ. Carlos III de Madrid
 Open for everyone
 Flipped the classroom methodology

18. 18
ANALYSE in the MOOC of maths
● Uses of ANALYSE in the experience
 Self-reflection for students
 Support for the flipped classroom
 Evaluation of educational materials
 Evaluation of students
 Evaluation of the course

19. 19
Example I: Self-reflection

20. 20
Example II: Evaluation of students

21. 21
Example III: Flipped classroom,
evaluation of materials

22. 22
Example IV: Evaluation of the course

23. 23
Present and future work in ANALYSE
● Design and implementation of higher level
learning indicators and their correspondent
visualizations
● Scalability. Work with a big amount of users
● Recommender based on previous data analysis
we have performed on other experiences ->
clustering, prediction, relationship mining
● Integration with Insights

24. 24
ANALYSE in the Spanish TV
● Part of the mapaTIC project:
 “La Aventura del Saber”, La 2, TVE,
http://www.rtve.es/alacarta/videos/la-aventura-del-
saber/aventuramapatic/3163463/
 ANALYSE and the MOOC of maths: 5:30-7:11

25. 25
WAPLA@EC-TEL
● Workshop on Applied and Practical Learning
Analytics
● Topics:
 Hands on Tutorial on exploratory data
analysis using Python and Spark, discussions
about LA tools, oral presentations
● Dates: Friday 18 September
● Place: Toledo (EC-TEL conference)
● More information
 http://educate.gast.it.uc3m.es/wapla/

26. Learning Analytics for Massive
Open Online Courses
Pedro J. Muñoz-Merino
Twiter: @pedmumeTwiter: @pedmume
email: pedmume@it.uc3m.esemail: pedmume@it.uc3m.es
Universidad Carlos III de Madrid