This document discusses using player rating systems to balance task difficulty in human computation games. It proposes treating tasks as players and using player rating algorithms to sequence tasks based on a player's changing skill level over time. The study tests whether a bipartite graph structure between players and tasks negatively impacts prediction accuracy of player rating algorithms. It finds that bipartiteness does not affect accuracy, and unbalanced graphs with "super vertices" may improve accuracy by providing more information. The approach shows promise for difficulty balancing, but requires further testing on retention and with different games.
CALL ON ➥8923113531 🔝Call Girls Kalyanpur Lucknow best Female service 🧵
Player Rating Algorithms for Balancing Human Computation Games: Testing the Effect of Bipartiteness
1. Player Rating Systems
for Balancing Human
Computation Games
testing the effect of bipartiteness
Seth Cooper, Sebastian Deterding, Theo Tsapakos
DiGRA 2016, August 6, 2016
c b
29. Research question
does a bipartite (player-player
or user-task) graph negatively
affect the prediction accuracy of
player rating algorithms? does
graph balancedness affect
accurcay?
38. main contributions
• Identified 4 challenges to difficulty balancing in human
computation games, crowdsourcing, UGC
• Introduced content sequencing through adapting player
rating algorithms as a novel approach
• Identified bipartiteness of user-task graph as potential issue
• Found that bipartiteness does not affect prediction accuracy
of ELO, Glicko-2, Truskill in Chess matches or human
computation game Paradox
• Found that unbalanced graphs improve prediction accuracy,
presumably due to super vertices/players
• Provided first support that our approach is viable
39. limitations & future work I
• Approach requires previous/initial data
• Use super-users to provide initial data
• Use “calibration” tasks in tutorials
• Use mixed method data to identify skill & difficulty indicators, data &
machine learning to validate & extract additional indicators
• Current algorithms only compute win/loss/draw
• Graded success measures could improve accuracy and learning speed
• Study trained on large data sets (10,000, 37 edges)
• Testing learning speed of algorithms w/ current default retention in human
computation games
• Study tested only one human computation game
• Replication with multiple games
40. limitations & future work II
• Study didn’t test direct effect on retention
• Follow-up user study
• Task pool might not contain tasks of best-fitting
difficulty (similar to empty bar in mulitplayer games)
• Procedural content generation to generate training/filler tasks
• Many human computation tasks don’t vary much in difficulty
• Expand matching approach to other factors like curiosity/variety