GIAF USA: Summer 2015
•Télécharger en tant que PPTX, PDF•
1 j'aime•3,599 vues
The Games Industry Analytics Forum returned for its third gathering of the year in San Francisco and its 10th meet-up, on Monday August 10th. Featuring presentations and expert panel discussions, the GIAF is a unique opportunity for practitioners looking to generate insight and value from big data game analytics; one of the most important trends in games. Talks at this GIAF: Nurturing the player journey Kady Srinivasan, Sr Director of Player Engagement - Mobile at Electronic Arts Lean analytics Will Perone, CTO at Wicked Fun Analytics architecture at IMVU Jon Watte, VP of Technology at IMVU Using data to prove the value of haptics Nick Thomas*, Head of Gaming at Immersion Corporation. The event is free for game analytics practitioners. For more info on future GIAF events, visit www.deltadna.com/GIAF *Slides from Immersion are not included in this slide deck at present
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à GIAF USA: Summer 2015
Similaire à GIAF USA: Summer 2015 (20)
Plus de deltaDNA
Plus de deltaDNA (8)
Dernier
Dernier (20)
GIAF USA: Summer 2015
- 2. UPCOMING TALKS Curating the Mobile Player Journey Kady Srinivasan- Sr Dir, Player Engagement Mobile at EA Lean Analytics Will Perone – CTO at Wicked Fun IMVU Data Architecture Jon Watte - VP Technology and Analytics at IMVU Using Data to Prove the Value of Haptics Nick Thomas - Head of Gaming at Immersion Corp
- 4. Curating the Mobile Player Journey Kady Srinivasan Sr Dir, Player Engagement EA Mobile
- 5. Agenda • Player Retention • Managing the Player Journey • Player Responsive Model • 4 Areas to support Player Responsive Model • Madden Mobile Rewards 2
- 6. Player Retention Key to Long Live Service Games Increasingly live service oriented games indicate heavy emphasis on engagement and retention over acquisition 0% 10% 20% 30% 40% 50% 60% ACQUISITION BRAND AWARENESS RETENTION Average Company Player retention is an integral part of product operations and EAM is beginning to invest Source: Return on Behavior %ofDollarsSpent 3
- 7. Managing the Player Lifecycle – Key to Retention MTX OFFER TUTORIAL YES 1. CONVERSION/ENGAGEMENT CYCLE MTX CONVERSION POINT 2 2. UPSELL/RETAIN CYCLE UPGRADE 3. AFFINITY X PROMO NEW PLAYERS NO Presenting relevant “need based” offers at key decision points in a player journey ENGAGEMENT OFFER INSTALLAFFINITY Opportunity: 4X increase in Retention and 2X increase in Engagement* *Source: Urban Airship
- 8. Curating Player Lifecycle – Four Key Areas Player Value (Who) Decision Points Mapping (When) Desired Action (Why) Customized Offers (What) • WHO: PLAYER SEGMENTATION: Classify player and expected value from player (i.e. Whales, Spenders, Social Influencers) • WHEN: DECISION POINTS MAPPING: Understand in- game and out of game behavior and decision points • WHY: DESIRED ACTION: Identify desired action we want the player to take • WHAT: CUSTOMIZED OFFERS: What offers fulfill need states TECHNOLOGY & DATA PLATFORM A scalable model of meeting player expectations supported by technology and 360 view of player Who -> When -> Why -> What 5
- 9. Who: Smart Segmentation Following Best Practice • RFM Segmentation • Based on Predictive Propensity • Multi channel Re-targeting and Customized Offers • In most games, players treated equal • Messaging not based on predictive propensity • All offers created equal • No multi channel marketing Generic Email Generic Promotions (Ex: 7 day Sales) BEHIND THE CURVE • Highly engaged: 3 sessions/day, > 10 mins per session, plays on iPAD • User has low probability of churn but high propensity to monetize • Show 10% interstitial offer on 2nd day, 50% on 4th day, push notification on 5th day, IAN on 10th day • High monetizer: 1 sessions/day, > 10 mins per session • User is fickle, convert early and high • Show new and exciting content ad on 2nd day, show how to use premium content, offer 50% off on 2nd purchase • Fickle user: < 3 sessions/week, < 10 mins per session • User is fickle, retain • New and exciting content, push notes on Day 3, 5 and 7 • Reminders to come back and play for new content through mobile notifications • If not engaging, X promo or show GMS ad CURRENT STATE Step function changes in targeted and segmented messaging in the last year 6
- 10. Who: Smart Segmentation V2.0 0 0.5 1 1.5 2 Push Note Response Predictors PNResponse RevenuePOST_PNResponse Predictors informed “personas” created from game play variables 0.0% 0.5% 1.0% 1.5% 2.0% 2.5% 3.0% $- $2.00 $4.00 $6.00 $8.00 $10.00 Friends Explorers Offerhogs Rushers DARPU & PN Response Rate by Persona DARPU PN_response % Personas FRIENDS EXPLORERS OFFER HOGS RUSHERS 7
- 11. Key: = PN = PN / IAM = IAM Green Text = New Campaign When: Key Points During Player Journey 1d Churn PN 3d Lapsed PN 5d Lapsed PN Job Complete PN Tutorial Complete IAM ~3x Status Quo Event- based Notes 7/30/60d Lapsed Spending Friend Visit IAM 7/30/60d Lapsed Spender 7/30d Single Transactor ~3x Status Quo Event- based Granpa Winback (Email Only) 14/30d Lapsed PN Merch Engine IAM / PN Promo Reminder PN Predictive Churn iAM / PN Lapsed Sage 7/30d PN New Player Mature Player Elder Player Player journey enhanced to drive greater touch points at relevant intervals 8
- 12. What: Need Based Offers RETARGET LAPSED PLAYERS FIRST TIME BUY OFFER PERSONALIZED PUSH NOTE REPEAT SPENDER OFFER eCRM PII COLLECTION NETWORK AFFINITY AD These messages are triggered at appropriate points in the player journey Right Message, Right Player, Right Time REPEAT MTX OFFER 9
- 13. Degree of customization What: Need Based Offers V 2.0 Provide compelling and differentiated offerings to influence the desired action Stage One All Players • Relevant and Personalized Messages • Promotions, Offers and Discounts • Limited and Unique Products/New Affinity Titles • Personalized Products/Store • Personalized Game Play Differentiated Experience Driven By: Relevant Messages Promotions Unique Products Personalize d Products Personalize d Game Play Stage Two Stage Three Stage Four Stage Five 130
- 14. 140 Summary Managing the player journey – massive opportunity but fraught with complexity Needs tight integration between Analytics, Marketing and Product Management to accelerate How can you unlock this potential?
- 15. Lean Analytics Will Perone CTO & Cofounder August 2015
- 16. Who Am I? • Designing and Programming games since age 11 • Making mobile games since 2005 • Making social games since 2007 • Worked for Glu, Funzio, Kixeye • Started 3 companies - Andrograde, RubyCoins, Wicked Fun
- 17. State of Wicked Fun • Venture backed • 16 employees • Cross-platform real time multiplayer game • Pre-revenue • Iterating the product fast • Iterating UA/Marketing fast
- 18. A Conundrum Be as lean as possible Iterate as fast as possible Execute the highest quality bar possible
- 19. Different Needs for Analytics Business Intelligence – Industry standard events easy – Hard to dig into the ‘why’ Design – Want complex game balancing metrics – Want in-game economy related analytics Engineering – Want performance/bug/crash monitors
- 20. Wicked Fun’s Solution • No dedicated PM/Analyst • Focus on a few core metrics per department • Don’t waste time building analytics in-house • Empower each team to do their own analytics • Balance quantitative and qualitative analysis, don’t let analytics run your life – Analytics can mislead you on what to focus on
- 21. Design • Use hierarchical events for balancing • Not all off the shelf analytics platforms support hierarchical events, may have to manually do it • Total in-game currency, Net currency over time • What are people buying in game • Certain things still require engineering to get involved (db queries, adding events to code)
- 22. Engineering • Use analytics as crash report statistics and performance/net monitoring tools – Ping, packet size, function times & counts… – Set up special API keys so you can turn it off if event traffic too high • Set up a different analytics API key per platform (mobile, web, PC…) – The platform itself will color user behavior even if the game is exactly the same
- 23. BI • Retention (D1, D7) – Connected to # people who convert to paying – How long does it take for people to buy? • Total funnel is hard to set up but worth it – View ->Install ->Register ->Tutorial ->Screen flow – Higher dropoff in your marketing or in your game? • What is the first thing people usually buy? • Organics vs Paid Installs -> Virality – Key to cheap UA
- 24. Industry Challenges There is still no good solution for cross-platform marketing, attribution and analytics Supporting infrastructure for real time multiplayer cross- platform games is still in its infancy Developers want real time analytics data feed to personalize UX to customers
- 27. IMVU Data Architecture Jon Watte, 2015-08-10 VP Technology and Analytics
- 28. Avatars Lean Start-up User Generated Content Continuous Deployment Virtual Currency Social Entertainment Social VR Multi-platform Real-time at Web Scale
- 30. Fun Facts 400 database shards for customers Haskell uses 10% as much CPU as PHP 20 million item catalog (works in VR, too!) Top selling UGC has sold > 1 million! IMVU creators pay the rent with UGC End-to-end analytics reduces payment fraud Recommenders give users what they want Price Elasticity optimizes revenue 12,000 new items per day 700,000 items sold every day300,000 metrics 10 second interval
- 32. JOIN IN THE CONVERSATION PARTICIPATE IN THE NEXT GIAF Analytics for Games events@deltaDNA.com www.deltaDNA.com/GIAF
Notes de l'éditeur
- \/ \/ \/ \/ \/ \/ \/ US (games) iPad US - #13 top grossing iPhone US - #32 top grossing Android - #44 top grossing Australia Who cares about Australia. We do iPad - #1 top grossing iPhone - #1 top grossing Android -#3 top grossing Overall, not only games
- Ad-hoc analytics, daily fires, dashboards, Insights Deep dive analysis - reports that take few weeks to complete; Predictive analytics, Machine learning, statistical modelling Data Pipeline, platform for machine learning and modelling
- Insights Data Science Data Engineering 7 people; 4 in London office
- We Are Hiring ! jobs@productmadness.com
- Events are generated on server-side. This way we control data quality. We are processing 350 Million events per day They got ingested into Amazon Cloud to S3, with the help of Python and Spark. And then got copied to Amazon Redshift - Columnar parallel database. Currently we have 12 nodes, with total capacity of 24TB Once the data is in there - we do all heavy aggregations and transformations. We have moved from Hadoop more than a year ago and haven’t looked back since. We perform most of interactive analysis in Python Notebooks. For trivial things we are using re.dash, which is similar to Mode and Periscope. It is Web-based SQL client with integrated plotting and collaboration functionality. You can even create dashboards with re.dash, but for production dashboards we prefer to use Tableau or our own D3.js-based application. All our web applications are using Python backend, Flask framework. We use scikit-learn for machine learning and predictive analytics. As you have probably guessed, we like Python.
- What we do: AB tests bread and butter of data science teams yet controversial and often misunderstood Customer Lifetime Value modelling knowing how much your customer worth, shortly after you acquite them is a holy-grail of User Acqusition can easily spend next 40 minutes talking about customer lifetime value modelling, but .. So - segmentation
- In this presentation I will not be taking in details about classification algorithm, dimensionality reduction or machine learning. Instead, we will be looking at segmentation from Product Marketing perspective.
- Successful segmentation is the product of a detailed understanding of your market and will therefore take time Segmentation is not a two-weeks task you assign to your analytics department
- Customers have different needs and means. Some players play for fun, others got a kick from competition. We all know that players have very different willingness to pay. Most of you know how rare it is to find a Normal Distribution among our players - our games are played by outliers. If you remove outliers from any analysis - you will probably miss the point of it. Segmentation can help to understand those differences Which can help to deliver on those needs And drive higher profitability
- A segment is a group of customers who display similarities to each other... They may react similarly in a product/service offering They may provide comparable values (profitability) to the company They may bear the same needs or behave in alike ways Customers move in and out of segments over time
- There is no one right way to segment (not should there be): Many different approaches and techniques. I will cover few techniques in the following slides. Mix of art, science, common sense, experience and practical knowledge You need to take business needs into account, but also what data is available and can be used, operationally. Don’t aim to build one holistic taxonomy to meet all needs, So what are different types of segmentation? How do you approach a problem like this?
- Multiple way to segment users And there are different use cases for segmentation. You can segment on: geography and basic demographics. In our case - Australian players are very important, and usually behave quite differently from the rest of the world. You can segment based on stage in a player’s Lifecycle - new players behave differently to someone who have been playing your game for the last two years. Also, knowing users who are showing signs of disengagement is very important. You can also segment on Behaviour, Needs (if you can identify them, possibly based on observed behaviour) and, of course, based on Player Value. But different parts of the business are interested in different segmentations.
- Product Managers and Marketing teams might be very interested in Behavioural Segments. But CEO may be more interested to track retention metric for your most valuable players (whales).
- The actual segmentation might be hybrid. This is the segmentation of the Land-based Slot players. First layer - by frequency of play, e.g. engagement Second layers - bahavioural
- But of course, it is important to understand why segmentation is useful for a business. What decisions can it help to make? And how it can affect daily operations and possibly product?
- Clients of Segmentation Strategy and Finance Product development Marketing operations
- Strategy and Finance
- When we looked at data after launch, amount of coints spent has actually dropped on the day of the launch! But was it even a real drop?
- But for a specific segment, that day was very positive.
- Business knowledge: - high-level segments goals - product/marketing strategy Data knowledge: - how to access 360 view? - what are segments definitions? Stats/Analytical skills - how to profile various segments? ETL - recalculated daily or real-time - regular reviews Integration with back-office and game - segmentation engine + ETL Dashboards Reporting Marketing: - day-2-day campaigns for segments - reporting (monthly and daily) Product - review feature success for segment Analytics and data engineering - ongoing support and refinement
- What are business objectives and therefore customer characteristics we should use to profile the market? What approach should we take to ensure segments accurately represent the market and actionable? What criteria should we use to prioritise segments and select targets? How can we ensure segmentation is operational and can be deployed? How to do ‘land’ the segmentation within the organization and ensure it gains traction?
- K-means Hierarchical Clustering Decision Trees .. and many more
- I’m going to take a look at what’s going on in game analytics. And why.
- I’m going to look at four aspects – what’s going on in the market we serve – in the games industry itself – and how that’s shaping what gets done in analytics. And I’m going to look at changes in the tools and tech we have at our disposal, to do the work we do. I’m going to look at frontier zones – areas what gets done is evolving. Liminal zones where the ocean and fog meet. Finally, I’m going to look at failure prevention in games analytics. (I didn’t want to leave you sad by titling it failure.) This is a glass half full take on things. I’ve only picked out a few things under each heading, and there might be more that you think of. Tell me in question time, or afterwards in the bar.
- You’re probably wondering. How does she know all this stuff? Has she wired my studio up for telemetry? Or is she making it all up? Well it’s neither. I do what you do – talk to people, read trade press, listen in on social media, look at changes in vendors’ service offerings, go to events, and use this info as tell-tales to see which way the wind is blowing. Last month I curated and chaired the Data Science and Analytics track at the games AI conference in Vienna, and I’m bringing back a few shiny snippets from that to share with you. Apparently crows don’t actually like shiny things, which I found out after researching this cool picture, but I do.
- It makes sense that what gets done in games analytics would be influenced by what’s going on in the games industry itself. So what’s going on in games? It depends a lot on what facet of the market you live in. The market as a whole is still growing. Could be worse. But there are two trends that make life difficult. They have to do with distribution – both on mobile appstores and on steam. One trend is the increasing number of games on the market. The other is the stickiness of the top 10 lists. Visibility in the face of competition, both long-tail and top 10, is a huge challenge. Best sellers tend to hang about like low cloud over England in summer. This isn’t by accident. It’s to do with store managers wanting to optimise their returns, and giving successful titles visibility via multiple internal channels. Also also, on mobile appstores, it’s about sophisticated use of the advertising ecosystem by top sellers. Also - the game needs to be good. But that isn’t enough.
- What’s happening as a result? It’s making people pay even more serious attention to distribution and visibility. Some are choosing to go with publishers rather than self-publish.
- Ouch.
- Really. Ouch. It’s bleeping expensive. This isn’t the kind of spend to do casually.
- This means having a good grip on your i/o for acquisition. This can get complicated. But the key point is that different players come in from different sources, which have different costs. You need to balance that view of your costs, with predictions about likely revenue. These predictions will become more accurate the longer a run of real data you have, but by that time your media buying window may have closed.
- Since it’s so nailbitingly pricey to acquire players, there is an increasing focus on understanding how to keep them. This has always been of interest. But the truism that usually cheaper to retain a customer than to get a new one is being taken more seriously, now that competition for attention is fiercer than ever.
- The areas that service providers are focussing on is often directional. App Annie has recently begun to offer competitor intelligence on how players interact with other games. After offering to integrating player metrics with their store performance data, for free.
- There’s lots to say about tech enablers. I’m only going to give a light kick to two aspects of it here – but ask me other things in the question time, or afterwards.
- One thing I’m seeing is the need for speed, for taking certain decisions. And with that an interest in streaming architectures and algorithms, particularly Spark streaming. There’s a good piece of work from nucl.ai on this, from one of our London games firms.
- I’d like to give a shout out to some open sourced work by Mind Candy, on using probabilistic data structures for stream-based metrics. This is very like the material covered in Ilya Katsov’s ‘Highly scalable’ blog, but it includes links to the source code. Most of these approaches use hashes to enable constant-space scalability, at the expense of perfect accuracy. Agrawal from Berkeley is the author to watch here, if this is your bag. He’s got a whole book on it.
- Also on the tech enablers front, there’s something almost unbearably hot in the machining learning world, that hasn’t yet become standard operating procedure in game analytics: deep learning.
- Actually this is a better picture. There’s huge excitement about deep learning as it enables the system to learn the features which are important – and not only that – learn a hierarchy of features, with lower level features being more general, and higher level features being category specific. This has resulted in big progress in speech recognition, and visual processing. The visual processing work is particularly interesting as it dovetails well with work on neurology of visual processing, and on mathematical modelling of processing channels.
- I have heard of people using it in analysis of play data, for player segmentation, but it’s been more along the lines of ‘I’ve tried everything but the kitchen sink and here’s this cool thing I will try too - I’m not quite sure what it’s good for but hey why not’. I’d say adoption is at the garage tinkering stage. Hence the picture of the messy machine shop. Which looks a bit like my desk. I’m not sure what problems in game analystics are the right shape, and have a strong analogy to signal processing. I think that’s tbd.
- By frontier zone, I mean places where practice is evolving, interesting, and not settled. You could I guess count the use of deep learning as a discovery technique as a frontier zone. Here are a few more.
- I’m not talking about Dark Side of the Moon, as the final frontier. But about being able to segment players (and game elements) in a way that informs design. There’s a particularly good piece of work in this direction from nucl.ai
- There has been a fair amount published on churn prediction detection – game play gives off a number of signals that can be useful in this respect. There was some interesting work at CIG2014 on that last summer, from Wooga in collaboration with University of Lausanne. What there’s been less of, is work that integrates churn detection with churn prevention. Here’s a really good piece of work from the track I chaired. What’s good about it combines analysis for prediction, for analysis to guide interventions. And it worked really well for them.
- Another frontier zone I’d wave a hand to is a meeting of minds between games user research and quantitive product management. Games user research is a discipline with its own set of conferences. I went to one in July, in London, and there’s going to be a big one, ChiPlay in London in October. In the whole day there was only one talk that had any numbers in it. It may not be obvious but these guys could make a great music together if they’d only learn each other’s languages.
- Finally here’s something everyone cares about. My poster child for this once again comes from someone else’s work: Meta Brown. She wrote Data Mining for Dummies. She’s not a dummy though. She has an advanced degree in nuclear engineering from MIT. She’s done lots of big ticket analytics project work, and consulting. She gave a talk earlier this summer at the Imperial College Data Science Institute. I’ve never laughed so hard in a data science talk. I can’t begin to imitate her dry midwestern sense of humour, but there was some good stuff she said that I think bears repetition.
- This is like the kind of koan you can chant while walking around whacking yourself on the head with a board. It’s that good.
- If you don’t get clear agreement about what success looks like – and to whom – success is going to be elusive. Hugely so. I’m not talking here about collecting basic metrics, but about going offroad, or on a deep dive in search of treasure.
- The key message here is to agree the business problem – and understand it as well as possible – before touching a drop of data. I do see people enjoying just diving in to see what’s there, and that’s fun for a side proejct, but being prepped property makes it easier to explore further. And as you get more complicated – you need more process support. That means Crisp-DM, or some cousin. According to Meta.
- Don’t go mad for big data when small data will do. Use it for what it’s good for.
- Here I’ve talked about what I think’s going on at the moment. As to what next, let’s take it to the bar.