This document discusses how gaming learning analytics can be used to improve serious games. It describes two EU projects, RAGE and BEACONING, that are developing frameworks to simplify incorporating learning analytics into serious games. RAGE is creating reusable technology components like game trackers and analytics servers. BEACONING aims to enrich the gaming learning analytics data model with contextual and location information. Both projects see potential for learning analytics to enable new forms of assessment and personalization in serious games.
1. Gaming Learning Analytics:
how projects RAGE and
BEACONING are improving
serious games applicability
Baltasar Fernandez-Manjon
balta@fdi.ucm.es , @BaltaFM
e-UCM Research Group , www.e-ucm.es
REV conference, Madrid, 26/2/2016
Realising an Applied Gaming Eco-System
http://www.slideshare.net/BaltasarFernandezManjon
2. Some definitions …
• Serious games Any use of digital games with purposes other than
entertainment (Michael & Chen, 2006)
• e.g. educational games, advertgames
• Some kinds of educational simulations could be serious games
• Gamification using game techniques into non gaming environments
• E.g. competition, collaboration, badgets, leaderboards, …
• Game Analytics is the process of capturing game interaction data
with different improvement purposes
• e.g. debugging, monetization
3. ¿Why DoD produce a free high-quality game?
America’s Army
3
http://www.americasarmy.com/
5. Remote laboratories and serious games?
KIBO – tangible programming – Tufts Uni.
KIBO is a robot kit specifically designed for young children aged 4-7 years old
The child creates a sequence of instructions (a program) using the wooden
blocks. They scan the blocks with the KIBO body to tell the robot what to do.
http://kinderlabrobotics.com/kibo/
6. More definitions …
• Learning Analytics Improving education based on data
analysis
• Data driven
• Evidence-based education
• Gaming Learning Analytics is a specific case when all
interaction data is used in serious games for improving the
learning process supported by the games
• Educational games not as “black boxes”
7. Learning analytics steps
• 1. Collecting large amounts of data from a number of channels
• Interaction with the system
• 2. Translating that data into information or actionable insights.
• It may be impossible to track how much a student really absorbed from one
lesson but the system CAN track his/her behaviour and use that as a signal
• 3. Use of the information for different purposes
• Personalization and adaptation. Once the system gets the signal, it can then
personalize each student’s learning environment.
• Assessment. Use that information for formative or summative assessment
• Predicting the best course in the future. As students use the system for a
prolonged period of time, educators will be able to track what works and
what doesn’t – and adjust accordingly.
http://www.edudemic.com/grades-2-0-how-learning-analytics-are-changing-the-teachers-role/
8. Uses of Gaming Learning Analytics in
educational games
• Game testing – game analytics
• It is the game realiable?
• How many students finish the game?
• Average time to complete the game?
• Game deployment in the class – tools for teachers
• Real-time information for supporting the teacher
• “Stealth” student evaluation
• Knowing what is happening when the game is deployed in the class
• Formal Game evaluation
• From pre-post test to evaluation based on game learning analytics??
9. Methodologies for serious game development
Torrente et at (2014) Development of Game-Like Simulations for Procedural Knowledge in Heathcare Education. IEEE Transactions on
Learning Tecnologies. 7(1), 69-82
USERS
In the medical domain
10. Formal evaluation pre-post
• Formal evaluation of games is very
complex and expensive
• Pre-test
• Post-test
• Very few games have been formally
probed to be effective
• Similar results with
Learning Analytics than
with pre-post test?
10
11. Can we use Gaming Learning Analytics for
formal evaluation of games?
Formal evaluation of
games from the
analysis of the user
(interaction) data?
14. H2020 RAGE in a nutshell
• RAGE will deliver advanced technology and know-how to support the
European Applied Games industry build-up and job creation
• February 2015, 4 years, 19 partners, 9M
• Creating a new serious games ecosystem by making available a set of
reusable technology components for developing advance serious
games easier, faster and more cost-effectively
• Open source gaming learning analytics framework
• provide all the required services (e.g. game tracker, learning analytics server,
visualization of analytics information)
• easy inclusion of gaming learning analytics techniques in the new games
Realising an Applied Gaming Eco-System
15. RAGE: creating the Gaming Learning Analytics infrastructure
H2020 RAGE project will simplify the process of SG
creation with ready to use assets
• Game trackers
• LA server infrastructure
• Standards support (e.g. xAPI)
Using interaction data for assessment
Realising an Applied Gaming Eco-System
16. Firsts results from RAGE: GLA support
Logical architecture
Actual architecture
17. GLA Asset Status in RAGE
• All available at github: code, documentation & demos, except
• 2.4c (User Model) server-side,
currently under collaborative development (first versión july 2016)
• 2.4b (Dashboard) plugin architecture,
which is under development - current dash-boards are hard-wired
• Additionally,
• rage-analytics project simplifies deployment of complex assets
• only dependency: docker & docker-compose
• platform and library agnostic
• Support for standards (xAPI)
• Host your server-side asset behind 2.4a (Auth & Authentication), enjoy
• single-sign on
• free HTTPs with your clients (via nginx and letsencrypt)
18. H2020 Beaconing project
• BEACONING stands for ‘Breaking Educational Barriers with
Contextualised, Pervasive and Gameful Learning’
• Started in january 2016, 15 partners, 9 countries, 6M
• Global goal is learning ‘anytime anywhere’
• Exploitation of technologies for contextual pervasive games and use of
gamification techniques
• Problem based approach to learning
• Enriching the Gaming Learning Analytics data model with
the contextual, geolocalized and accessibility information
• Large pilots in real settings
• Formal and informal learning across virtual and physical spaces
19. Gaming for good – games or labs?
• gaming approaches applied to solving complex problems in a
collaborative way
19
Play to Cure™: Genes in Space - a mobile game in which players collaborate to analyse real genetic data
(Cancer Research UK, n.d.)
http://centerforgamescience.org/portfolio/foldit/
http://www.cancerresearchuk.org/support-
us/play-to-cure-genes-in-space
20. Conclusions
• LA in Serious Games has a great potential from the application and
research perspective
• Still complex to implement LA in SG
• Increases the (already high) cost of the games
• New specification could simplify the task (e.g. xAPI)
• New open frameworks produced by EU projects and new standards
specifications could greatly simplify GLA implementation and
adoption
• And this is just the beginning (BYOD, VR, new interactions devices..)
• Games are driving the disruption ….
20
1. Collecting large amounts of data from a number of channels – including, but not limited to, online learning environments, social, mobile – and perhaps in the future, games. Couple this data with various learning theories and we can begin to form a more holistic picture of a student’s learning progress than just theories.
2. Translating that data into actionable insights. It may be impossible to track how much a student really absorbed from one lesson but the system CAN track his/her behaviour and use that as a signal. Here are a few examples of behavioural signals:
- Language of frustration in any media.- Low time on site, relative to the class.- Long lag between logins.- Tracking areas of studies in which the student is weak in over years.- Detecting the TYPE of mistakes that was made – careless or a fundamental lack of understanding?- Theoretically, learning analytics would even be able to track whether or not a student is guessing in a multiple choice test.
3. Personalization and adaptation. Once the system gets the signal, it can then personalize each student’s learning environment. For example, if a student spends significantly less time attempting to solve a problem compared to other students, the system can display prompts and clues to keep him/her going – in real time. This is crucial because when a student gets feedback is just as crucial to learning as what feedback the students get. This wasn’t possible in the past, where students have to wait at least a few days for their assignments to be marked.
4. Predicting the best course in the future. As students use the system for a prolonged period of time, educators will be able to track what works and what doesn’t – and adjust accordingly. In fact, it will soon be possible for each student to essentially be working with a custom-built and personalized curriculum that’s unique to them.
Game development approach
Involves domain experts from the very beginning
Usually selecting cases to be used in the game (from cases/problem based teaching to game story. This process is natural to medical personnel
Agile and iterative development methodology
Analysis: scrip -> description of the procedure
Game design: game elements + game mechanics
Implementation: incremental game versions, from mocks-up to final versions
Quality assurance: checking with experts if the game version meet the initial requirements