EICS 2023 presentation of https://dl.acm.org/doi/10.1145/3593223. Abstract: Although a vast number of mobile health interventions has been discussed in the scientific literature, the scientific evidence for engineering gamified health interventions is too meager. Depending on the context, this leads to immature investments on one side of the spectrum and delayed implementations on the other side. GameBus is a web platform for increasing the rigor of gamified health experiments in living lab settings. This article clarifies the scientific design rationale of this platform and provides an eight-year evaluation of its performance. We demonstrate that a large variety of studies has been performed, leading to a promising data set for analyzing the impact of gamification techniques in the context of health promotion. While various articles on such studies have been published, along with data sets, this article shares novel information to enable other scholars to scale up such efforts. We also highlight critical gaps to guide future work on the platform.

1. 8-year Evaluation of GameBus: Status quo in Aiming for
an Open Access Platform to Prototype and Test
Digital Health Apps
@EICSCONF, SWANSEA, JUNE 28TH, 2023
Pieter Van Gorp and Raoul Nuijten
Images and icons from Unsplash and FlatIcon

2. TL;DR;
What is the problem?
The scientific evidence for engineering gamified health interventions is
too meager
2

3. TL;DR;
Who is impacted by that problem?
This leads to immature investments on one side of the spectrum and
delayed implementations on the other side
3

4. TL;DR;
What is our contribution?
Solution: a web platform for increasing the rigor of gamified health
experiments in living lab settings, based on emerging design criteria
4

5. TL;DR;
What do we elaborate on?
Design rationale and 8-year progress assessment
5

6. TL;DR;
Impact of solution/contribution?
(a) demonstrating stronger evidence base, and
(b) enabling others to strengthen it further
(c) path to ML/RL-based personalization
Future: clear indication of critical gaps and results are open access
6

7. Example topics calling for more experimental
control and academic efficiency
• Effects of badges as rewards
• Effects of doctor-provided advices as rewards
• Effects of monetary rewards
• Effects of lottery mechanics
• Effects of social newsfeeds
• Effects of involving family members
• Effects of involving colleagues
• Effects of involving teachers
• Effects of autonomy in health game rules
• …
7

8. Sketch of our playing field
societal burden
of non-communicable
diseases
8
Preventable diseases
correlated with
lifestyle
Scalability potential of
mHealth
Poor adherence to
mHealth interventions
Need for better
evidence base comprehensive set of
GameBus-based
mHealth insigts
GameBus as
scientific mHealth
platform
(gamification focus)
Generalizability of the
design criteria beyond
the mHealth field
Design
Criteria???
Platform extensibility
beyond Gamification

9. So…
9
This is NOT:
To lecture you on the burden of:
• lifestyle diseases,
• mental health issues,
• challenges in adherence to
medication, rehabilitation
training, etc.

10. And…
10
This is NOT:
• A story of yet another mHealth app
• About a new gamification strategies,
new persuasive health techniques,
new visualizations,
• …

11. Instead, our research question:
How can one design a ``system of systems'' to experiment
with socialization and gamification concepts in living lab
settings, to produce scientific knowledge and data while
taking into account vague and conflicting requirements such
as privacy preservation and openness?
11

12. Remainder of the talk
• First, the result
• Then, how we got there
• Finally, the suggested road ahead
12

13. The result in one slide
13

14. Research Methods
Practice Reviews
Literature Reviews
Conceptual Modeling
UML Use Case Diagrams, Class Diagrams
User Interaction Models
Software Architecture Models
Business Models
Living Lab Experimentation
API Design
14
Practice Reviews
Literature Reviews
Conceptual Modeling
UML Use Case Diagrams, Class Diagrams
User Interaction Models
Software Architecture Models
Business Models
Living Lab Experimentation
API Design
Practice Reviews
Literature Reviews
Conceptual Modeling
UML Use Case Diagrams, Class Diagrams
User Interaction Models
Software Architecture Models
Business Models
Living Lab Experimentation
API Design
8 Years
Various funded projects
Dozens of design iterations

15. Research Methods
Criteria for 8-year Evaluation:
 Flexibility
 Input Interoperability
 Output Interoperability
 Valid Metrics on Health & Wellbeing
 Valid Metrics on Engagement
 Replicability
 Evidence Base Size
 Knowledge Base Size
 Openness
 Privacy Preservation
 Sustainability
15

16. Results
16
#PracticeReviews
#ConceptualModeling
Enabling social scientists to
mimic various state-of-the-
practice app characteristics by
means of configuration on a
common basis
Paving the way to ML, RL-based
personalization, JITAIs

18. 18

19. 19

20. 20

21. Results
21
#Architecture
#ConceptualModeling

22. 22
Architecture that supports studying Multi-dimensional Gamification
(inspired by practice reviews, generalization of BOOSTH)
#SystemOfSystems

23. minigames.gamebus.eu
app.gamebus.eu
api.gamebus.eu
GameBus
database
gis.gamebus.eu
GameBus Cloud
Google Fit
FitBit
Participant Consent
minigame.thirdparty.app
Researcher Desktop
Participant Consent
#architecture
23

24. Results
24
Criteria for 8-year Evaluation:
 Flexibility
 Input Interoperability
 Output Interoperability
 Valid Metrics on Health & Wellbeing
 Valid Metrics on Engagement
 Replicability
 Evidence Base Size
 Knowledge Base Size
 Openness
 Privacy Preservation
 Sustainability

25. Featured Result Evaluation 1: Flexibility
• Flexibility Level 1: the GameBus Challenge Editor (adapt game rules)
• Flexibility Level 2: use of GameBus Studio (enable/disable mechanics)
• Flexibility Level 3: Mini-Game Framework (include custom widgets)
• Flexibility Level 4: extension of GameBus Studio (add custom pages)
• Flexibility Level 5: Custom Front-end
25

26. Featured Result Evaluation 2: Evidence Base Size
26
(>30 entries)

27. Featured Result Evaluation 2: Evidence Base Size
27

28. Road Ahead
28
Criteria for 8-year Evaluation:
 Flexibility
 Input Interoperability
 Output Interoperability
 Valid Metrics on Health & Wellbeing
 Valid Metrics on Engagement
 Replicability
 Evidence Base Size
 Knowledge Base Size
 Openness
 Privacy Preservation
 Sustainability
Based on Pathverse and
Mobistudy: strengthen data
dashboard features for peer
scientists
So far, API exports and BI
tooling had been
sufficient
DPIA Templates

29. Concluding
What is our contribution?
Solution: a web platform for increasing the rigor of gamified health
experiments in living lab settings, based on emerging design criteria
Impact of solution/contribution?
(a) demonstrating stronger evidence base, and
(b) enabling others to strengthen it further
(c) path to ML/RL-based personalization
29
 Flexibility
 Input Interoperability
 Output Interoperability
 Valid Metrics on Health & Wellbeing
 Valid Metrics on Engagement
 Replicability
 Evidence Base Size
 Knowledge Base Size
 Openness
 Privacy Preservation
 Sustainability

30. << BACKUP SLIDES >>
30

31. Screenshots: Flexibility Level 1
31

32. Screenshots: Flexibility Level 2
32

33. Screenshots: Flexibility Level 3
33

34. Screenshots: Flexibility Level 4
34

35. Screenshots: Flexibility Level 5
35