In this talk we will analyze the effects of gamification in the social network of a large online course on ‘digital skills for teachers.’ Educational social networking websites and learning systems can gather information about contributions of participants and about the underlying social network. We will present an experimental gamification layer with three game elements (points, badges, and leaderboard) that was delivered to students. Social network analysis (SNA) and principal component analysis (PCA) can then be used to analyze the differences between groups using information about contributions to the website, and position and influence in the social network of each participant. Initial results suggest that variables and participants group differently, and that gamification may influence the structure of the social network of participants in the course. The first component (F1) can be a good descriptor of students’ work and position in the network that can be used to build predictive models of learning success. The models suggest that the probability of passing the course increases more rapidly in the experimental (gamified) groups for students that participate.

1. Social network analysis of large
online gamified courses
Luis de Marcos Ortega, Ass. Prof.
Computer Science Department
Universidad de Alcalá
luis.demarcos@uah.es
https://goo.gl/sBjvw1
1
Time4Science. July 2018. Varaždin
Faculty of Organization and Informatics. Varaždin. Univ. Zagreb.

2. Index
• Introduction
• Background and related work
• Objectives
• Methods
• Results
• Discussion
• Conclusions
• References
2

3. Introduction
• Participants
3

4. Background and Related Work
Gamification
• Gamification is the use of game elements in non-game
contexts to promote participation and motivate action
(Deterding et al., 2011; Werbach and Hunter, 2012).
• Framing activities as a game through game elements,
like points, badges, and leaderboards, holds as much
psychological power as the full game mechanics
(Lieberoth, 2014).
• Gamification increases users’ performance for simple
repetitive tasks (Mekler et al., 2013) but findings of its
motivational effects are contradictory (Hanus and Fox,
2015; Mekler et al., 2017).
• Effectiveness is also in question, pointing to the
necessity to align gamification with the goal of the
activity and to address psychological needs of users at
design time (van Roy and Zaman, 2017).
4

5. Background and Related Work
Gamification in education
• Educators are trying to harness the potential of
gamification to design motivating learning
experiences.
• Education is the most common context in which
gamification is implemented and reported (Hamari et
al., 2014; Seaborn and Fels, 2015; Martí-Parreño et
al., 2016).
• Existing research presents mixed results regarding the
impact of gamified elements such as badges, points,
and leaderboards in learning and affective outcomes.
• For instance positive effect on practical assignments but
a negative effect on conceptual learning (Domínguez et
al., 2013)
5

6. Background and Related Work
Social Networking in Education
• Important limitation: most of the existing
research on the utility and effectiveness of social
media in higher education is limited to self-
reported data (e.g., surveys, questionnaires) and
content analyses (Tess, 2013)
• Educational social networking sites provide data
for analysis, like the contributions of participants
and connections between them  Social network
analysis (SNA)
• Application of SNA to e-Learning environments is
at a very early stage, although the number of
studies is increasing (Cela et al., 2015)
6

7. Background and Related Work
Gamification + Social Networking (in Education)
• The number of studies that bring together
gamification and social networking is limited
• Social gamification framework to assist teachers
in creating motivational learning experiences
fitted to learners’ needs (Simões et al., 2013)
• No analysis of results
• In a series of studies de-Marcos et al. compared
gamification and social networking concluding
that both yielded similar results regarding
learning performance in an undergraduate
course (de-Marcos et al., 2014; de-Marcos et al.,
2016a; de-Marcos et al., 2016b)
7

8. Objectives
1. Scrutinize the structure of the underlying social
network in large online courses
2. Analyze the effect of gamification in the structure of the
social network on large-scale online courses
3. Study the impact of position & influence (in SN),
contributions (in SNS) & game elements on learning
success in large-scale online courses
• Predictive models of student success
8

9. Setting
• Undergraduate online course “Digital Skills for
Teachers” (free offering in Portuguese)
• MOOC approach
• autonomous and independent learning with a
strong emphasis on the social and collaborative
dimension.
• 4 ECTS. 6 weeks. Syllabus:
1. Searching and sharing online resources
2. Using digital tools in the classroom
3. Promoting collaborative learning using digital
tools
• 3 editions: 363, 427 and 591 students
9

10. Methods
Instruments
• Social networking educational gamified platform
10

11. Methods
Instruments
•Functionality SNS:
• Social networking (all groups)
• news, learning guide, dashboard, blogs,
bookmarks and internal twitting
• Gamification layer (experimental group)
• thirteen achievements, points, and a
leaderboard
•Technical implementation: Moodle + Elgg
11

12. Methods
Measures
• Social Network Analysis (SNA)
• Network measures can be used to analyze the
social interactions and the structure of the
network, as well as changes over time
• Hypothesis: Gamification influences contributions
and social network
• Measures of each individual participant
represent her position and influence in the
network
• Effects of gamification at the level of each individual
• Analyze learning performance in relation to
contribution, position and influence in the network
12

13. Methods
Measures
• Individual network metrics
• Degree, Closeness centrality, Eccentricity, Betweenness
centrality, Clustering coefficient, Eigenvector centrality
• Link analysis: PageRank, authority, and hub
• Overall network metrics
• Avg. degree, Graph density and Avg. path length
• Participation metrics
• blogs, tweets, likes, messages (to other participants),
comments (to any publication), followers, following,
logins and total interactions
• number of achievements and points earned
(experimental group only)
• Learning performance: Passed or not passed
13

14. Methods
Tools
•Principal Component Analysis (PCA)
•Graph visualization (Gephi)
•Nodes: participants
•Edges: connections (followers)
•Only central connected component
•Regression
14

15. Results
Structure of the social network
• Experimental vs Control
15

16. Results
Structure of the social network
16
Experimental Control
Nodes (including teachers) 600 437
Edges 3200 715
Avg. degree 5.33 1.64
Graph density .009 .004
Avg. path length 2.29 2.59
Avg. clustering coefficient .49 .36
Nodes with degree<>0
(at least 1 connection)
313 (53%) 167 (39%)
Students that passed 31 (5.25%) 14 (3.28%)

17. Results
Predictive models of learning success
• F1 (first component) is good a descriptor of
global student activity:
• Experimental group
• Includes 18/20 variables (measures)
• Describes 68% of variability
• Control group
• Includes 16/18 variables (measures)
• Describes 55% of variability
• Use F1 as predictor of learning success
17

18. Results
Predictive models of learning success
18
y = 0.0346x + 0.0525
R² = 0.3335
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
-5.000 0.000 5.000 10.000 15.000 20.000 25.000 30.000
Certificate
F1

19. Results
Predictive models of learning success
•Problems (MOOCs…):
• Many of the students that registered carry
out little or no work at all
• Dichotomous variable (passed/not
passed) to measure learning success
• Students that do a lot of work (activity) do
not earn the certificate
• Students with little activity pass
19

20. Results
Predictive models of learning success
• For better charaterization  Introduce a new
measure called success probability (probability of
getting the certificate)
• Order the dataset by F1 (or any other predictor variable)
• sample the values above and below a given value taking
n values that create a sliding window of samples
• where A(j) is success at point j (A(j)=1) or not (A(j)=0)
20

21. Results
Predictive models of learning success
21
y = 0.0512x - 0.0228
R² = 0.8974
-0.10
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.000 5.000 10.000 15.000
ProbCertificate(n=21)
F1
Experimental

22. Results
Predictive models of learning success
22
y = 0.0282x + 0.0346
R² = 0.9107
-0.05
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.000 2.000 4.000 6.000 8.000 10.000
ProbCertificate(n=21)
F1
Control

23. Results
Predictive models of learning success
• Probability of getting the certificate increases more
rapidly in the experimental group for students that
engage getting higher scores in the variables that
are part of F1
• Gamification seems to mediate in learning success
through increased participation
• Game elements > Participation > Success
23

24. Discussion
• Gamifaction fosters connections / many nodes with
an important number of connections (hubs).
• redistribution in the flow of communication between
students that widens and changes the patterns of
participation among participants (Aviv et al., 2003)
• Personal activity and structural centrality in the
educational social network are correlated (Klein et
al., 2015).
• Variability explained by the first component (F1) is
higher for experimental group
• gamification contributes to explain students’ work,
providing a better statistical description
24

25. Discussion
• Previous studies point to correlations between
social networking and learning success (Cho et al.,
2007; Thoms, 2011), and between gamification-
driven social networking and learning success (de-
Marcos et al., 2017; de-Marcos et al., 2016b) which
our study confirms (MOOC course).
• F1, besides being a good representation of
student’s work and position in the network, it is
also a good estimate of the probability of success.
25

26. Conclusions
• Gamification influences the final structure of the
social network as measured by network metrics
and individual connections of participants
• Network metrics and measures of participation are
also a good representation of student work that
facilitate building predictive models of the
probability of success for students
• Predictive models show that students in the
experimental condition (gamification) have a higher
probability of passing the course getting a
certificate if they participate.
26

27. Limitations
• Causal relationship between the effect of social
networking and gamification in learning
performance is not proven
• Quasi-experimental design
• Generalization
• Three cohorts of students and three social networks are
studied
• Particular educational setting
27

28. Future Work
• Confirm effect of gamification in
participation
• e.g. clustering / reduction of dimensionality
• Alternative data analysis (e.g. data mining)
• Mobile learning and augmented reality
• Motivations & player types
• Other constructs/scales (e.g. competition vs
collaboration)
28

29. References
AVIV, R., ERLICH, Z., RAVID, G. & GEVA, A. 2003. Network Analysis of Knowledge Construction in Asynchronous Learning
Networks. Journal of Asynchronous Learning Networks, 7, 1-23.
CELA, K. L., SICILIA, M. Á. & SÁNCHEZ, S. 2015. Social Network Analysis in E-Learning Environments: A Preliminary
Systematic Review. Educational Psychology Review, 27, 219-246.
CHO, H., GAY, G., DAVIDSON, B. & INGRAFFEA, A. 2007. Social networks, communication styles, and learning
performance in a CSCL community. Computers & Education, 49, 309-329.
DE-MARCOS, L., DOMÍNGUEZ, A., SAENZ-DE-NAVARRETE, J. & PAGÉS, C. 2014. An empirical study comparing
gamification and social networking on e-learning. Computers & Education, 75, 82-91
DE-MARCOS, L., GARCIA-LOPEZ, E. & GARCIA-CABOT, A. 2016. On the Effectiveness of Game-like and Social Approaches
in Learning: Comparing Educational Gaming, Gamification & Social Networking. Computers & Education, 95, 99-113.
DE-MARCOS, L., GARCÍA-LÓPEZ, E., GARCÍA-CABOT, A., MEDINA-MERODIO, J.-A., DOMÍNGUEZ, A., MARTÍNEZ-HERRÁIZ,
J.-J. & DIEZ-FOLLEDO, T. 2016. Social network analysis of a gamified e-learning course: Small-world phenomenon and
network metrics as predictors of academic performance. Computers in Human Behavior, 60, 312-321.
DE-MARCOS, L., GARCIA-CABOT, A. & GARCIA-LOPEZ, E. 2017. Towards the Social Gamification of e-Learning: a Practical
Experiment. International Journal of Engineering Educadion, 33, 66-73.
DETERDING, S., DIXON, D., KHALED, R. & NACKE, L. 2011. From game design elements to gamefulness: defining
"gamification". Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media
Environments. Tampere, Finland: ACM.
DOMÍNGUEZ, A., SAENZ-DE-NAVARRETE, J., DE-MARCOS, L., FERNÁNDEZ-SANZ, L., PAGÉS, C. & MARTÍNEZ-HERRÁIZ, J.-J.
2013. Gamifying learning experiences: Practical implications and outcomes. Computers & Education, 63, 380-392.
HAMARI, J., KOIVISTO, J. & SARSA, H. 2014. Does Gamification Work? – A Literature Review of Empirical Studies on
gamification. 47th Hawaii International Conference on System Sciences. Hawaii, USA. 29

30. References
HANUS, M. D. & FOX, J. 2015. Assessing the effects of gamification in the classroom: A longitudinal study on intrinsic motivation,
social comparison, satisfaction, effort, and academic performance. Computers & Education, 80, 152-161.
KLEIN, A., AHLF, H. & SHARMA, V. 2015. Social activity and structural centrality in online social networks. Telematics and
Informatics, 32, 321-332.
LIEBEROTH, A. 2014. Shallow Gamification: Testing Psychological Effects of Framing an Activity as a Game. Games and Culture.
MARTÍ-PARREÑO, J., MÉNDEZ-IBÁÑEZ, E. & ALONSO-ARROYO, A. 2016. The use of gamification in education: a bibliometric and
text mining analysis. Journal of Computer Assisted Learning, 32, 663-676.
MEKLER, E. D., BRÜHLMANN, F., OPWIS, K. & TUCH, A. N. 2013. Do points, levels and leaderboards harm intrinsic motivation?:
an empirical analysis of common gamification elements. Proceedings of the First International Conference on Gameful Design,
Research, and Applications. Toronto, Ontario, Canada: ACM.
MEKLER, E. D., BRÜHLMANN, F., TUCH, A. N. & OPWIS, K. 2017. Towards understanding the effects of individual gamification
elements on intrinsic motivation and performance. Computers in Human Behavior, 71, 525-534.
SEABORN, K. & FELS, D. I. 2015. Gamification in theory and action: A survey. International Journal of Human-Computer Studies,
74, 14-31.
SIMÕES, J., REDONDO, R. D. & VILAS, A. F. 2013. A social gamification framework for a K-6 learning platform. Computers in
Human Behavior, 29, 345–353.
TESS, P. A. 2013. The role of social media in higher education classes (real and virtual) – A literature review. Computers in Human
Behavior, 29, A60-A68.
THOMS, B. 2011. A Dynamic Social Feedback System to Support Learning and Social Interaction in Higher Education. IEEE
Transactions on Learning Technologies, 4, 340-352.
VAN ROY, R. & ZAMAN, B. 2017. Why Gamification Fails in Education and How to Make it Successful: Introducing Nine
Gamification Heuristics Based on Self-Determination Theory. In: MA, M. & OIKONOMOU, A. (eds.) Serious Games and
Edutainment Applications, Volume II. Chan, Switzerland: Springer.
WERBACH, K. & HUNTER, D. 2012. For the win: How game thinking can revolutionize your business, Philadelphia, Wharton
Digital Press.
30

31. Social network analysis of large
online gamified courses
Luis de Marcos Ortega, Ass. Prof.
Computer Science Department
Universidad de Alcalá
luis.demarcos@uah.es
https://goo.gl/sBjvw1
31
Time4Science. July 2018. Varaždin
Faculty of Organization and Informatics. Varaždin. Univ. Zagreb.