Gamification: How Effective Is It?

Aucune remarque pour cette diapositive

WHY:It’s on the rise – entire companies devoted to provision of gamification services (Bunchball, BigDoor, Gamify.com); forecasted to be a big portion of social marketing budgets in futureThe promise of gamification is that the simple addition of game-like elements can increase participationand engagement in any process or experience. We set about to test such claims, particularly as they pertain to market research. To do this, we conducted a review of the literature, interviewed people at the forefront of the gamification trend, reflected on an experiement we conducted for a previous paper on gamification, and hunted for additional data and case studies to support/refute the efficacy claims

AcknowledgementsWe would like to thank the many people who generously gave up their time to answer our questions with their invaluable input, including:Jesse Schell, gamification guru and Chief Executive Officer and Creative Director of Schell GamesSebastian Deterding, gamification guru and PhD researcher in communication science at the Graduate School of the Research Center for Media and Communication, Hamburg University Michael Wu, Principal Scientist of Analytics at Lithium Technologies Danny Day, CEO of QCF Design (IGF award-winning, independent game development house)Rolfe Swinton, Director of Lumi MobileJon Puleston, Senior Director of GMI InteractiveFrancesco D’Orazio, Research Director and Head of Social Media of Face GroupPhil Groman, Head of Innovations for AfroesKevin Spier, Director of Sales, and Dan Maier of BunchballBo Nielsen, researcher with TNS Gallup Denmark

Video gamesIt is important to make the distinction upfront between what we traditionally mean by “games” (e.g. video games, board games, party games, etc.) and a “gamified” experience. Gamification does not entail the turning of a traditionally serious experience (such as filling out your tax return or a research survey) into a gaudy, frivolous entertainment experience. An experience does not need to look overtly like a game in order to be compelling and engaging. For example, Facebook is probably the most successful non-game game ever invented. It has implemented game mechanics and reward structures that keep a large portion of the world’s population coming back several times a day. Yet, Facebook has no mascot, complimentary colours, traditional level structures or anything else that we would usually associate with a game.Badgification/pointsificationWe also don’t mean the simple addition of badges, points and bright colours – the mere inclusion of these things does not automatically make an experience fun and engaging. This type of shallow design is sometimes pejoratively referred to as “badgification” or “pointsification”.

Gamification , as we define it, refers to the ‘gamifying’ of traditionally non-game processes and experiences through the purposeful introduction of mechanics that are designed to elicit specific, predictable behaviours, while simultaneously absorbing individuals in the experience by making it engaging and compelling.

One of the most commonly cited examples of gamification is Farmville, the massively successful online social game, in which players are coaxed toward particular behaviours through various game mechanics, e.g. the ability to collect items (something we instinctively love to do); a social element that rewards players for roping their friends into the game; a competition element that works by fostering envy for what players on neighbouring farms have accomplished; etc.Farmville is a fairly obvious example of gamification in action. It looks like a game, feels like a game…. It is a game.

Foursquare is a more subtle application of game mechanics. Foursquare is a location-based social networking site that rewards users for reporting their location via GPS. The user who checks in most frequently to a given location is awarded the status of mayor of that location. This looks a little like a game, but isn’t actually a game in itself. It’s more like a game layer imposed on top of an existing behaviour

An even more subtle (i.e. not overtly game-like) example is that of Klout, a web service that measures one’s online influence by pulling data from your Twitter profile, Facebook etc. You are given a Klout score, which can then be compared to that of your peers. Klout goes beyond measuring and also tries to influence, for instance, by offering perks for increasing your Klout. Klout is by no means a game, but there are game mechanics at play.

How does gamification work?It works by tapping into our ingrained biases and tendencies, essentially hijacking the brain by pushing the buttons that create sensation and rewardGamification taps into the deep-seated, primal tendencies and biases that have been sculpted over time by evolution to maximise our chances of survival. 10,000 years ago, we used these deeply ingrained biases and tendencies to effectively navigate our natural environment. Today, games have become adept at pushing these same buttons to create sensations of enjoyment and reward. Tom Chatfield [2010] identifies several ways in which games push our buttons: Experience systems give us a sense of achievement for reaching milestones and keep us working towards them. For example, the LinkedIn progress bar shows how much profile information you still need to provide and outlines the simple steps you need to take to achieve that goal.Short- and long-term goals in parallel allow us to achieve on a micro scale while simultaneously making progression towards the macro goal seem more manageable. Rewards for effort (i.e. positive reinforcement) trigger releases of feel-good chemicals in our brain, which train us towards desired behaviour. For example, Foursquare rewards users with badges for checking in the most times at a specific venue (mayor badge).Rapid, frequent, clear feedback in response to a user’s actions which also sets off the reward centres in our brains. For example, Facebook is addictive partly because it allows its users to receive real-time feedback in response to their comments and Likes.An element of uncertainty is crucial for an effective reward scheme. Gamblers become addicted to slot machines due to the unpredictable nature of the pay-offs [see Montague & Berns, 2002]. It is the element of uncertainty that has people constantly checking to see whether an email has arrived in their Inbox or whether someone has commented on their Facebook status.Other people (i.e. social elements) probably provide our brains with the greatest rewards. Humans are social creatures by nature. Adding other people into your experience is a no-brainer as evidenced by the rise of social media in the form of Facebook, Twitter, etc. According to Gavin Marshall, Head of Innovation for Mxit, South Africa’s largest social network, MXit users’ rewards are mostly social in nature.

Examples of such mechanics might include: Achievement: People like to win and have a feeling of control or mastery over their actions. This is a very basic human drive that can be harnessedAppointment dynamic: “A dynamic in which to succeed, one must return at a predefined time to take some action.” e.g. happy hour at your local pub [Priebatsch, 2010]Communal Discovery: “wherein an entire community is rallied to work together to solve a riddle, a problem or a challenge. Immensely viral and very fun.” [Priebatsch, 2010]See Appendix 1 in the written paper for an extensive list of game mechanics.

So, can anything be gamified? Sebastian Deterding is a prominent gamification “guru” who has done much research into what he calls ‘purposeful design’. He recommends that one first assesses the social norms and conventions surrounding an activity to ascertain if fun and playfulness are things that will be appreciated by the community, or if it is a context in which they would be frowned upon. Furthermore, he suggests that one ought to “understand why people engage in a behaviour - and to figure out whether ‘motivation’ is really the issue.

Typically, the objective of gamification is enhanced user engagement with the brand, product or service.Regrettably, this is a goal that many ‘gamified’ experiences fall short of attaining. This is attributable toone or more of several factors...

Jesse Schell quoteWhen tasks are directly tied to material rewards, constant calculation of the anticipated payoffs can diminish interest in the activity itself. Rewarding behaviour that would otherwise be done for free can backfireD’Orazio quote:Enhancing engagement should not come at the expense of data integrity. One needs to consider the potential forbiasing the results through ‘nudging’ and weigh this up against the expected gains in engagement. Given that there is potential to manipulate how people respond, things like benchmarking then become important.

When trying to use game mechanics to influence behaviour, there are limits to how much one can realistically control, giving way to the possibility of undesirable emergent behaviours. An example is that of BMW’s location-based game prototype intended to encourage fuel-efficient driving by challenging drivers to beat fuel efficiency records set by others driving the same route. While the game achieved its purpose of lowering the fuel consumption of test drivers, undesirable driving behaviours like skipping red lights emerged, as drivers realised that stopping and then accelerating again used more fuel [Deterding, 2010]. Unintended behaviour can also surface when participants try to manipulate the system to maximise the payoff for themselves – a phenomenon referred to as ‘gaming the system’ [Deterding, 2010]. Strategic Synergy [2010b] recommends building in a mechanism for continual feedback (through analytics, for instance) to enable the designer to evolve the game mechanics in such instances.

To get to the bottom of the actual effectiveness of the trend, we previously conducted an experiment to see what effect, in terms of user engagement, the addition of game mechanics can have on an online community [Alberts & Findlay, 2011]. In the experiment, we set up two visually identical online groups, which were both posed with the same question (“What makes the perfect T-shirt”) designed to stimulate conversation. The only difference between the experimental and control group was that the experimental group included game mechanics, while the control group did not. The experimental group’s mechanics included a progress bar, parallel goals, badges, a virtual currency and an appointment dynamic. We discovered clear differences in engagement levels between the groups (see Figure 1 for a summary of the results which showed a clear increase in engagement amongst the experimental, gamified group). Specifically, we found that the gamified, experimental group……had more members who contributed responses (83% vs. 68% of members)…were more likely to start discussions, as a greater proportion of posts were in response to other members’ answers rather than directly to our structured questions (37% vs. 3% of posts were comments in response to other members’ answers)…had a higher average number of posts per participant (2.3 vs. 1.5)Unfortunately, due to the relatively small size of the group, it was not possible to take the results as clear-cut evidence for gamification’s effectiveness, regardless of how compelling the results appeared.

Perhaps the best example of gamification being successfully applied to an online community is that of community architects, Lithium Technologies’, execution for giffgaff, a simcard-only mobile virtual network in the UK. Rather than employing a traditional support team, giffgaff relies on its community members for customer support and other functions. They do not have a call centre. Members are rewarded with “kudos” which elevate their personal reputation and with “Payback Points” once every six months which can be used to pay for their mobile services, exchanged for cash or donated to a charity. Kudos and Payback Points are earned for everything from answering community questions, promoting the company or attracting new members. Lithium’s implementation has been so successful for giffgaff, that the mobile company, which only has 16 full-time staff members, has a customer satisfaction score of 91% - unprecedented for most organizations. In addition, it has been nominated for several awards, including the Forrester Research International Groundswell Award and the UK Brand of the Year award. For the purpose of this paper, we interviewed Michael Wu, Lithium’s Principal Scientist of Analytics, who attributes much of giffgaff’s success to the gamification elements integrated into giffgaff’s community platform, which contribute directly to community members’ rewarding experiences, keeping them coming back and contributing time and again. The statistics speak for themselves:Over 10,000 questions were asked in the ‘Help’ forums in 2010 and the community responded with over 100,000 answers100% of the questions were answered by the communityThe average question response time is within three minutes, 24 hours a day, seven days a week95% of queries are answered within 60 minutesOver 40% of customers contributed to the community in the first six months Put simply, giffgaff offers an unprecedented business model. The company is run by the community for the community on the back of a well-gamified experience. According to Wu, “The effect of gamification is pretty astounding and has even surprised us in what it’s able to do in the case of giffgaff”. The giffgaff experience clearly supports our own previous experimental results, which showed the incredible effectiveness of gamifying an experience.

Kevin Spier and Dan Maier of Bunchball shared another case study of gamification effectiveness with us. Bunchball integrates game elements into websites and non-traditional game media. They partnered with NBC Universal to revamp the fan site for their comedy series, Psych. Bunchball added several game elements to the site in order to achieve this goal, including allowing fans to accumulate points for completing tasks such as watching videos, solving puzzles and listening to songs. These points could be redeemed for prizes such as T-shirts, mugs and autographed set memorabilia. In addition, high-scoring fans competed with each other via an online scoreboard. The results again seem to bear testament to the value of creating a compelling online experience through game elements:Overall site traffic increased by 30%Page views increased from 9 million the season before to 16 million in the last, gamified seasonThe average visitor came four to five times a month instead of just twice a monthThe average time spent on the site increased from 14 minutes to 22 minutesOnline merchandise sales increased by 47%

Finally, with regards to community curation, website building service, DevHub, launched an updated, gamified version of their website in conjunction with gamification service provider, BigDoor. The new site awarded users with points for adding additional elements to their site. As users earn more points, they gain access to new options for customizing their website. They could also fast-track this process by purchasing these features for real money. The gamification approach proved very effective in terms of both increased engagement and revenue:Prior to gamifying the website, users would complete four actions before logging out (perhaps forever). Post-gamification, the average user completed twelve actions during their first sessionIn addition, twice as many new users now post to their blogs (up from 7% to 15%)Prior to relaunch, virtual goods were a negligible part of the site’s revenue. Post-relaunch, they made up 29% of gross revenues

Playboy, also in partnership with Bunchball, ran a gamified campaign on Facebook designed to attract young subscribers. The campaign in question was Playboy’s Miss Social campaign. Women were encouraged to submit their photos for votes, the goal being to garner more votes than anyone else. They were given a variety of activities that they could engage in to generate more votes. The results were dramatic:85% users played multiple times, and over 50% returned the next month60% increase in revenue via the game from one month to the next

Another type of marketing amplification that we came across uses actual video games to deliver a message. These game experiences are commonly referred to as “advergames”. We chatted to South African company, Afroes, about the use of branded games designed to increase a brand’s engagement with customers and the adoption of a specific message. In addition to their work with commercial brands, Afroes have also created a game experience for the Nelson Mandela Children’s Fund entitled Champ Chase. The aim of the game is to educate young people about child abuse, children’s rights and what to do if they encounter an abusive situation. To measure the effectiveness of the game, Afroes conducted a focus group consisting of twenty children, aged 11 to 15 (16 girls and 4 boys) in Jabulani, Soweto, South Africa. Two evaluation worksheets were completed, before and after playing the game. According to Phil Groman of Afroes, “The objective was to establish whether the player’s knowledge and understanding of child protection issues had changed as a result of playing the game, and if so to what degree.”Despite the small sample, and while they do not give us an indication of the long-term effect of playing the game, the results were still impressive. They found that, on average, participants’ awareness levels increased by 32% after playing the game (based on their composite measure). Other findings included:Only 15% mentioned Childline (an abuse hotline) before playing the game; 95% after playingOnly 20% were able to identify a type of child abuse pre-game; 85% identified sexual abuse, emotional abuse, physical abuse, child trafficking or neglect after playing

Lumi has explored how game elements can motivate people to complete demanding research tasks, such as regularly completing a detailed behaviour diary. For example, the same study was conducted amongst two separate groups, one in the USA and one in the UK. Participants were required to enter their product consumption roughly every hour or so for ten days via mobile device. The UK group had a very standard survey diary design. The USA mobile application included key game elements like progress bars, colourful and friendly icons, and elements of uncertainty such as different kinds of reminders and varying frequencies of reminders, depending on the user's consistency in completing the diary.The difference between the groups was quite striking. The UK group, which used a more traditional application without game elements, saw about 30% completion rates and were less happy with the experience, rating it around 3 or 4 out of 10. Conversely, the USA group had a nearly 100% completion rate and an average happiness of 8 out of 10. This appears to be a clear indication again of the effectiveness of including game elements.

Lumi also gamified research for the Oscars awards. In that study, participants played along on their mobile phones, voting on performances, commenting on fashion and rating the hosts. This gamified methodology kept participants watching 42% longer than non-players. Furthermore, players rated their enjoyment of the Oscars 50% higher than non-players. It also focused more attention on the sponsors. While sponsor recall during the show was similar for both player and non-player viewers, recall two days after the show among players was higher at nearly 100%. This seems to indicate that game-like experiences are better at activating attention and ensuring that people engage in a more lasting way.

Finally, a variation of the mobile television research methodology was used for the most recent Eurovision Song Contest. We chatted to Bo Nielsen of TNS Gallup Denmark who conducted this study. While the results were still in an early stage of processing at the time of writing, Nielsen was very happy with the results and the methodology’s implications for market research in general. Participant comments that came out of the Eurovision study bear testament to this, including comments such as: “It was like being a part of a community”“Noticed more details in the show than normally”“Fun to be a part of this new kind of tests”“Really liked the chat, where people discussed the show” “Found it funny to rate the TV-show and see the results” Nielsen considers the game-based methodology very useful and an improvement on traditional methodologies, like CATI, CAWI and CAPI, all of which make more onerous demands of participants. Perhaps the biggest benefit of the approach was an increase in engagement, which, in turn, had spill-over effects in terms of data quality. Indeed, participants were so engaged in the experience that they exhibited unintended behaviour such as turning the experience into a drinking game and returning to the application two days after the test to ask the research company for more questions and to continue chatting with other users. Swinton points out that this is a common pattern seen with this methodology. Participants proactively reach out to find when the next event will be. According to Swinton, “This points to the fact that creating the right game experience is very appealing and motivating. People feel part of something valuable”.

What could we do to make surveys more engaging in future?To begin with, researchers would do well to pay more attention to aesthetics and visual design. With companies like GMI, Conquest, Annik Technology Services and Lumi Mobile developing software to overcome the technology barriers, it is likely to become much easier for researchers to make surveys interactive without any prior technical know-how. Tarran [2011a], for instance, describes how GMI’s Qstudio technology includes several different ways of approaching a grid-style question without using a traditional grid, such as asking participants to drag-and-drop flags along a scale.Gamified design can be as unadorned as reframing the language of questions to make them more interesting for participants to answer. For instance, Puleston [2010] has shown how one can achieve greater depth of response by asking participants to imagine what they would order as their last meal if they were on death row, rather than simply prompting them to name their favourite food. He maintains that we limit the depth of insight that can be gained through adherence to standard survey questions like ‘Indicate your level of agreement with this statement on a scale of 1 to 10’ [Tarran, 2011a]. BrainJuicer’s predictive markets technique is an example of successful reframing. Numerous predictive markets experiments have shown how asking participants to play the role of a venture capitalist with money to invest in product ideas can improve the quality of the data obtained. Reframing questions as challenges is another simple way to introduce a powerful intrinsic motivator. People are naturally drawn to experiences of challenge and mastery. Puleston [2010] describes how time-based challenges - such as giving participants two minutes to write out a shopping list of all the food items they would stock up on if shopping for an entire year - generates more thorough responses than a traditional question.

For the most part, traditional survey design is anything but engaging. Approaches to questioning can be painfully formulaic, and with the possible exception of a progress bar here and there, surveys are not concerned with providing prompt feedback to those answering them. Typical incentives take the form of material rewards – the least compelling of all motivators, and also the type most likely to be seen byparticipants as cheap manipulation. Clearly, we have a long way to go.Some of the sluggishness of the industry in designing more interactive surveys is due to technological barriers, rather than a lack of willingness. There are companies like GMI Interactive who are investing a lot in overcoming those technological barriers.What types of things can we do to make surveys more engaging?Visual design: we would do well to pay more attention to aesthetics. For example, there are alternative ways of approaching traditional grid-style questions, like asking participants to drag and drop flags along a scaleReframing the language of questions: it can be as simple as reframing the language of questions to make them more interesting for people to answer, e.g. Puleston has shown how one can achieve greater depth of response by asking participants to imagine what they would order as their last meal if they were on death row, rather than simply prompting them to name their favourite foodRapid, clear feedback could certainly be more widely utilised in surveys. Jesse Schell noted in our interview that “When people care about what they are doing, and get good feedback about it, their engagement is more serious, and certainly more sustained”. One way in which this is currently done is with progress bars, which tap into our basic need to complete things. Expanding upon this, one could wrap up each section of the survey with an indication of how the participant’s answers compare to those of others.We could also look for ways to build in uncertainty ,novelty and intriguee.g. through unlocking a new piece of a jigsaw puzzle with each section of the survey that is completedbuild in status in online research communities by awarding badges for the number of surveys completed, or for introducing new members to the panel. Particularly whenthese badges can be carried across other websites and social networks, they serve as a way of promoting one’s online reputation and influence, and signifying one’s interests and brand affinities. As long as badges and points are included in a meaningful way and as partof a broader design, they can be instrumental in conferring the elusive reward of status.Incorporate social elements: traditional surveys are a solitary affair. While it would be challenging to build in social elements into surveys, cracking the challenge would lead to more engaging (and thus better) surveys

What does gamification bring the market research table? **Deterding quote***Possibly the greatest challenge of gamification within market research is for us, as researchers, to step outside of our comfort zones. Since the complex and somewhat regimented nature of the traditional survey doesn’t lend itself toward creating experiences of autonomy and mastery, achieving meaningful engagement often requires a dramatic overhaul in approach. **Danny Day quote**

While there is certainly a lot of hype and controversy around gamification at the moment that could warrant it being labelled a fad; there are also lasting and meaningful things at the core of it which we can learn and benefit from…

Gamification, if done properly, does appear to work. That said, gamification is no magic elixir and there is no one success formula.The research industry can learn a lot from gamification, and the potential benefit of gamifying surveys appears to be vast. Capitalising on this potential will require more than just the technology to build nteractive surveys; it will require a readiness on the part of the researcher to depart from traditional ways of interviewing and a commitment to step into the shoes of the participant with a view to buildinga truly interesting and engaging experience.

EPILOGUE: AcknowledgementsAgain, we would like to thank the many people who generously gave up their time to answer our questions with their invaluable input, including:Jesse Schell, gamification guru and Chief Executive Officer and Creative Director of Schell GamesSebastian Deterding, gamification guru and PhD researcher in communication science at the Graduate School of the Research Center for Media and Communication, Hamburg University Michael Wu, Principal Scientist of Analytics at Lithium Technologies Danny Day, CEO of QCF Design (IGF award-winning, independent game development house)Rolfe Swinton, Director of Lumi MobileJon Puleston, Senior Director of GMI InteractiveFrancesco D’Orazio, Research Director and Head of Social Media of Face GroupPhil Groman, Head of Innovations for AfroesKevin Spier, Director of Sales, and Dan Maier of BunchballBo Nielsen, researcher with TNS Gallup Denmark

Gamification: How Effective Is It?

Socialphysicist

Gamification: How Effective Is It?