23_11_2018 Seminario eMadrid sobre «Analíticas para el aprendizaje» organizado por la UAM

1. Implicaciones de datos de aprendizaje
procedentes de MOOCs
Seminario eMadrid sobre «Analíticas para el
aprendizaje
Madrid, 23 de noviembre de 2018
Edmundo Tovar Caro
etovar@fi.upm.es
Universidad Politécnica de Madrid

2. Learning Analytics (LA).
Processes
Mohammad Khalil & Martin Ebner, TU Graz

3. The Environment
MOOCs and Open Education

4. A way of carrying out education, often using digital
technologies. Its aim is to widen access and participation to
everyone by removing barriers and making learning accessible,
abundant, and customisable for all. It offers multiple ways of
teaching and learning, building and sharing knowledge. It also
provides a variety of access routes to formal and non-formal
education, and connects the two
Open Education is seen as
Source: JRC IPTS Report: Opening up Education: a support framework for higher education institutions.
http://publications.jrc.ec.europa.eu/repository/handle/JRC101436
Open Education
CC BY Jonatan Castaño Muñoz
Jornada red.es-UC3 sobre formación digital con tecnología abierta

5. Research Lines
MOOCs and Open Education
Who enrolls?
Impact on Higher Education
How is the learning?

6. New measurements needed
— Different audiences than traditional HE degrees
— Different intentions and motivations
Source: online course report State of the MOOC 2016: A
Year of Massive Landscape Change For Massive Open Online
Courses
CC BY Jonatan Castaño Muñoz
Jornada red.es-UC3 sobre formación digital con tecnología abiert

7. For whom?
— Main actors in this scenario: MOOCs providers and EC
— To develop policy guidelines: issues to learn more about
the population of MOOC learners
— Benchmarking: reports will provide MOOC providers insigths
about their MOOCs compared with other MOOCs

8. New measurements needed

9. Research Lines
MOOCs and Open Education
Who enrolls?

10. Profiles of MOOC participants
ü MOOC format is characterized by the great diversity
of enrolled people
ü There is a lack of knowledge of participants’
background
ü It is needed to provide a real picture of the
heterogeneity of MOOC participants

11. — Geographic distribution
— There are students from 117 countries. The country with
more students is USA with 17.4% of the total, followed by
Spain (6.9%) and India (6.8%).
1st MOOC of the IEEE Education
Society
edX analytics

12. — MOOCKnowledge online pre-questionnaire
— Sociodemographics:
— gender, date of birth, socioeconomic status (highest degree
or level of school completed, employment status (currently
and two years ago)).
— Previous experiences with MOOCs:
— MOOCs related to participants’ current job, number of
completed MOOC with certification and kind of certificates
obtained.
— Usefulness of the MOOC:
— professional, personal and academic life.
Data set

13. — The scenario is fairly
uniform in the period
of two years until the
start of the course.
— Participants with a
job are almost three
quarters of the
dataset in both
periods of time
Sociodemographics. Employment status
(currently and two years ago)
PROFESSIONAL STATUS
Professional status
Time window
current (%)
2 years ago
(%)
Employed 102 (73%)
102
(76.7%)
self-employed 12 (8.6%) 11 (8.3%)
unemployed and looking
for a job
9 (6.4%) 2 (1.5%)
unemployed but not
currently looking for a
job
1 (0.7%) 0
a student (not in the
labour force)
8 (5.7%) 12 (9%)
in retirement or early
retirement
6 (4.3%) 4 (3%)
permanently disabled 1 (0.7%) 0
a home maker (fulfilling
domestic tasks)
1 (0.7%) 2 (1.5%)
TOTAL 140 133

14. — Most of them are aware of the MOOC’s
usefulness on their professional sphere
Goals and intentions. Usefulness of the MOOC for the
participants’ professional, personal and academic
life
Usefulness of the MOOC for the professional, personal and academic life of the participants (n = 119))

15. Extended profiles of MOOC participants
— ANALYSIS: Approach 1: Data
mining

16. Study case
— Scenario:
— Analysis of the behavior of students attending the different levels of
involvement to the MOOC who have registered
— Groups of interest: managers in open education, academic staff of
MOOCs,…
— Objective:
— To increase the size of the target cluster in next editions of the MOOC
— Phases:
— 1. Clustering of participants
— 2. Target cluster?
— 3. Identifying profile of students “bridge” between the target cluster
and other clusters
— 4. Promoting the participation of “bridge” students in the future
— ANALYSIS: Approach 2: SNA

17. Study case.
Clustering of groups of participants
— The SNA analysis is based on the
similarity of participant profiles:
if two participants give the same
answer to a question, a
relationship exists between
them.
— The network consists of 316
nodes (participants) with 42117
relationships pointing.
— We obtained three clusters
represented with different
colors.
— Each cluster represents the
pattern of participants that have
the tendency to answer of the
same way.
HOW?
SNA metrics and filters:
• Weight (link)
Clustering: modularity metric method
After applying multiple filters to the network the number
of nodes is reduced to 306 and the number of edges to
23216.

18. Research Lines
MOOCs and Open Education
Impact on Higher Education

19. Creating a SPOC from an OCW
in Higher Education
— Improvements in the student
perceptions of materials and
resources
— The elimination of the final exam.
Assessments in peer-to-peer format
and applied rubrics for assessment.
— Short videos and answer
questionnaires, instead of
attending long traditional classes.

20. Research Lines
MOOCs and Open Education
How is the learning?

21. The influence of different teaching
modes on learner s learning behaviors
— In pure MOOCs
— In flipped clasroom modes
— In traditional classroom

22. Thank you!
Edmundo Tovar Caro (etovar@fi.upm.es)

23. MOOCKnowledge project
Initiative launched by the Institute of Prospective Technological Studies (IPTS).
Executed by the Open University of the Netherlands (OUNL) in cooperation with the Open
University of Catalonia (UOC) and Universidad Politécnica de Madrid (UPM)
MOOCKnowledge pursues to assess the current perspective of
learners as participants of European MOOCs: when, why and
how the participants enrolled in one or more MOOCs take
advantage of them
Goal: building a [large scale] and cross-provider data collection
based on the experiences reported by the participants of the
MOOCs offered by providers that collaborate with the project
with the intention of a more deepen understanding of issues
related to MOOC format
Online multilingual
survey:
• Pre-questionnaire
• Post-questionnaire
• Follow-up questionnaire
Pre-questionnaire:
• attibutes:
demographics,
linguistics, motivation,
intentions, LLL, MOOC
experience, prior
satisfaction of
participants with MOOC
format

24. Clustering

25. Clustering

26. Analysis
Approach 1: Data mining

27. Integrated regulation: tendency to try
new things, to keep up to date, aligned
with training needs
Identified regulation: contribution of
value, considered useful, meaningful and
beneficial
Extrinsic regulation: avoid feelings of
ashamed of myself or guilt
Amotivation: no desire to do the MOOC,
it is considered a waste of time, no priority
for the participant and he/she would prefer
to do other things
Extended profiles of MOOC participants
Motivational perspective
Introjected regulation: behavior to avoid
guilt or to enhance the ego

28. The features that make up the degree of
prior satisfaction of participants are …
• course materials
• MOOC presentation
• video lecture
• MOOC website
• teacher(s)
• communication with organizers of the
MOOC
• learning activities
• discussions in the MOOCs forum
• assignments
• quizzes
• feedback given to participants
• pace of the MOOC
• language used
• difficulty level of the MOOC
• needed technical requirements
• offered certifications options
Prior satisfaction perspective
Extended profiles of MOOC participants

29. Satisfied (790 participants) Indifferent (405 participants) Dissatisfied (50 participants)
MOOC presentation (0,941)
Language used (0,934)
Course materials (0,920)
Learning activities (0,909)
Technical requirements (0,906)
MOOC website (0,885)
Teacher(s) (0,880)
Video lecture (0,866)
MOOC level (0,856)
Assignments (0,837)
MOOC pace (0,792)
Quizzes (0,768)
Communication organizers (0,719)
Feedback given (0,680)
Certification options (0,642)
Forum discussions (0,573)
Assignments (0,815)
Feedback given (0,807)
MOOC pace (0,783)
Quizzes (0,775)
MOOC level (0,738)
Certification options (0,723)
Learning activities (0,711)
Teacher(s) (0,709)
Forum discussions (0,709)
Communication organizers (0,689)
Technical requirements (0,662)
MOOC website (0,657)
Video lecture (0,652)
Course materials (0,644)
MOOC presentation (0,635)
Language used (0,627)
Course materials (0,800)
MOOC presentation (0,800)
Assignments (0,780)
Quizzes (0,780)
MOOC level (0,780)
Video lecture (0,780)
MOOC website (0,780)
Language used (0,760)
Feedback given (0,740)
Technical requirements (0,740)
Teacher(s) (0,720)
MOOC pace (0,700)
Communication organizers (0,700)
Learning activities (0,700)
Certification options (0,680)
Forum discussions (0,573)
Preliminary findings: prior satisfaction
+ No Experience with MOOC format (337 participants)

30. 74 participants with high languages
proficiency, reasonable motivation before their
enrollment in a new course, and without
previous experience in MOOC format
214
211
212
Preliminary findings: extended profiles
104 participants with high languages proficiency,
reasonable motivation before their enrollment in a
new course, and a high degree of previous
satisfaction in MOOC format
53 participants with high languages proficiency,
reasonable motivation before their enrollment in a
new course, and indifference regarding MOOC
format

31. Analysis
Approach 2: SNA
PhD Student: Jorge López

32. Exploration of SNA to the
Moocknowledge data
— An initial study has been conducted on the potential of
social network analysis (SNA) and its metrics for:
— Analyzing data obtained from the answers the participants.
— Exploring useful scenarios.
— The data source used comes from 715 participants of
the MOOC on Test Anxiety Management.

33. Study case.
1. Clustering of groups of participants
— Cluster 1 (Green color): Students always finish the MOOC
including all the assignements and assessments
— Cluster 2 (Red color): There is diversity of levels of
involvement in its members
— Cluster 3 (Blue color): Students finish the MOOC very often
doing all the activities planned

34. Study case.
2. Target cluster?
— Supossed:
— Target cluster is number 2
— Objective:
— In future editions of the MOOC we seek to increase the target
cluster size, preserving its characteristics, through the
participation of cluster 1.

35. Study case.
3. Identifying profile of students
“bridge”
— Profile of students contributing to the objective
— Second stage of tertiary education
— Employed for wages
— Woman
— Spanish
— Born in the 1960s
— Living together with partner
— Working in the Public administration and defense; compulsory
social security sector
HOW?
SNA metrics and filters:
Betweenness