Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Building a Data-Driven Culture
Olof Hoverfält
Director, Sanoma Games
September 2, 2015
@Hoverfalt
Sanoma Games is a diverse yet compact and lean unit
B2C & B2B, web & mobile Autonomous unit Key functions “in-house”
Busin...
DESIGN
BUSINESS ANALYTICS
UX 
Business
Being data-driven requires much more than the right tools
http://toolmonger.com/category/85/computer/page/2/
Handheld
USB ...
...Design Product Marketing Business
Being data-driven requires much more than the right tools
Organization supporting lea...
...Design Product Marketing Business
Being data-driven requires much more than the right tools
Organization supporting lea...
http://theleanstartup.com/
http://www.startupolic.com/launching-your-service-think-about-ux/; https://en.wikipedia.org/wiki/Lean_startup
An MVP is “a...
http://www.yesware.com/blog/how-to-close-more-deals-with-lean-email/
A continuous Build-Measure-Learn cycle enables fast l...
...Design Product Marketing Business
Being data-driven requires much more than the right tools
Organization supporting lea...
A data-driven culture cannot be created by coercion, but must
be built through intrinsic motivation from its visible benef...
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
Improvement actions
xxx
Revenue
ROS&packag.
campaign vol.
Video
campaign vol.
Pric...
Adoption of a data-driven approach requires for people to see
its benefits; how they could kick ass even more at what they...
Ownership
is not about doing what I tell you to do
is not about not doing what I tell you not to do
is about doing what I ...
...Design Product Marketing Business
Being data-driven requires much more than the right tools
Organization supporting lea...
Data-drivenness, at both individual and team level, is a
capability that can and should be developed
http://skepdic.com/te...
A core principle of learning is to be hypothesis-driven. Random
experimentation will yield various outcomes but little ins...
Data without insight can be destructive as it may lead
development the wrong way look like improvement
DAU
(daily active u...
Main KPIs should be static but reflect the current growth phase,
while tactical tools can be whatever helps improve the KP...
...Design Product Marketing Business
Being data-driven requires much more than the right tools
Organization supporting lea...
A simple tool that is actually used provides infinitely more value
than a sophisticated one that is not
https://xplr.com/x...
...Design Product Marketing Business
Being data-driven requires much more than the right tools
Organization supporting lea...
Goals set at function level can often be unaligned or conflicting,
breaking joint effort and leading to sub-optimal perfor...
...Design Product Marketing Business
Being data-driven requires much more than the right tools
Organization supporting lea...
...Design Product Marketing Business
Being data-driven requires much more than the right tools
Organization supporting lea...
Example:
Continuous improvement and A/B testing
A/B testing is about continuous and systematic gradual
improvements that over time build increased performance
2.5x+
Impro...
Seemingly small but sustained improvements can have a
very large impact on overall performance
49%
Improvement
min to max
...
Example:
Retention (how well you keep your users)
6.1%
Retention D1
Count of users by level
Count of users by level
Count of users by level
6.1%
Retention D1
47%
Retention D1
25%
Retention D7
6.2%
Retention D90
Compare with
6.1% D1 Retention
in the previous exam...
Example:
LTV ((Customer) Life Time Value)
User profitability is basically determined by LTV* and CAC*.
Being able to even roughly predict LTV can be very insightful...
Example:
Liigapörssi (KPIs and tools)
Direct
Org. search
Email
SEM
Display
Other
Our main time unit of comparable comparison is a game
“phase”. The key desired ...
We use various tools to identify development opportunities, run
tests and measure impact on the main high-level KPIs set
A...
Some tools we are currently using
Tool Domain Sought benefits
Email Improve open rate and CTR
Email (transactional) Identi...
Some suggested further reading
* http://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-version
Slideshare*
...Design Product Marketing Business
Being data-driven requires much more than the right tools
Organization supporting lea...
@Hoverfalt
Building a data-driven culture
Building a data-driven culture
Building a data-driven culture
Building a data-driven culture
Building a data-driven culture
Building a data-driven culture
Building a data-driven culture
Building a data-driven culture
Prochain SlideShare
Chargement dans…5
×

sur

Building a data-driven culture Slide 1 Building a data-driven culture Slide 2 Building a data-driven culture Slide 3 Building a data-driven culture Slide 4 Building a data-driven culture Slide 5 Building a data-driven culture Slide 6 Building a data-driven culture Slide 7 Building a data-driven culture Slide 8 Building a data-driven culture Slide 9 Building a data-driven culture Slide 10 Building a data-driven culture Slide 11 Building a data-driven culture Slide 12 Building a data-driven culture Slide 13 Building a data-driven culture Slide 14 Building a data-driven culture Slide 15 Building a data-driven culture Slide 16 Building a data-driven culture Slide 17 Building a data-driven culture Slide 18 Building a data-driven culture Slide 19 Building a data-driven culture Slide 20 Building a data-driven culture Slide 21 Building a data-driven culture Slide 22 Building a data-driven culture Slide 23 Building a data-driven culture Slide 24 Building a data-driven culture Slide 25 Building a data-driven culture Slide 26 Building a data-driven culture Slide 27 Building a data-driven culture Slide 28 Building a data-driven culture Slide 29 Building a data-driven culture Slide 30 Building a data-driven culture Slide 31 Building a data-driven culture Slide 32 Building a data-driven culture Slide 33 Building a data-driven culture Slide 34 Building a data-driven culture Slide 35 Building a data-driven culture Slide 36 Building a data-driven culture Slide 37 Building a data-driven culture Slide 38 Building a data-driven culture Slide 39 Building a data-driven culture Slide 40 Building a data-driven culture Slide 41 Building a data-driven culture Slide 42 Building a data-driven culture Slide 43 Building a data-driven culture Slide 44 Building a data-driven culture Slide 45 Building a data-driven culture Slide 46 Building a data-driven culture Slide 47 Building a data-driven culture Slide 48 Building a data-driven culture Slide 49 Building a data-driven culture Slide 50 Building a data-driven culture Slide 51 Building a data-driven culture Slide 52

Les vidéos YouTube ne sont plus prises en charge sur SlideShare

Regarder la vidéo sur YouTube

8

Partager

Télécharger pour lire hors ligne

Building a data-driven culture

Télécharger pour lire hors ligne

Video recording available here (YouTube): http://bit.ly/1KAmiCq

Building a data-driven culture

  1. 1. Building a Data-Driven Culture Olof Hoverfält Director, Sanoma Games September 2, 2015
  2. 2. @Hoverfalt
  3. 3. Sanoma Games is a diverse yet compact and lean unit B2C & B2B, web & mobile Autonomous unit Key functions “in-house” Business development Marketing Design Development Sales
  4. 4. DESIGN BUSINESS ANALYTICS UX  Business
  5. 5. Being data-driven requires much more than the right tools http://toolmonger.com/category/85/computer/page/2/ Handheld USB Snake Scope
  6. 6. ...Design Product Marketing Business Being data-driven requires much more than the right tools Organization supporting lean development Data-driven culture Key concepts Accessible tools Shared goals Management fostering self-direction Key concepts Key concepts
  7. 7. ...Design Product Marketing Business Being data-driven requires much more than the right tools Organization supporting lean development Data-driven culture Key concepts Accessible tools Shared goals Management fostering self-direction Key concepts Key concepts
  8. 8. http://theleanstartup.com/
  9. 9. http://www.startupolic.com/launching-your-service-think-about-ux/; https://en.wikipedia.org/wiki/Lean_startup An MVP is “a version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort" The MVP approach is useful at all levels of development, from single product feature to new business concepts Adopt MVP thinking in all development
  10. 10. http://www.yesware.com/blog/how-to-close-more-deals-with-lean-email/ A continuous Build-Measure-Learn cycle enables fast learning but is really sensitive to friction in workflow and cooperation Ensure all key functions are fully dedicated and the workflow is frictionless
  11. 11. ...Design Product Marketing Business Being data-driven requires much more than the right tools Organization supporting lean development Data-driven culture Key concepts Accessible tools Shared goals Management fostering self-direction Key concepts Key concepts
  12. 12. A data-driven culture cannot be created by coercion, but must be built through intrinsic motivation from its visible benefits Transparency Autonomy Ownership
  13. 13. xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx Improvement actions xxx Revenue ROS&packag. campaign vol. Video campaign vol. Pricing Discounts Campaign volume Optimization Direct Referral SEO Ret. visitors Potential issues Site problems Ad blockers New visitors Reach Monetization Visits Premium fill rate Premium eRPM Remnant eCPM Unique visitors Media sales breakdown by contributing component Impression success rate nn% nn% nn%  mm% nn% nn% nn% lost Ensure overall understanding of how the business works and of the contribution of and the dependencies between its components. Make deep data available In order for a team to develop and fine tune an engine, everyone must know how it works
  14. 14. Adoption of a data-driven approach requires for people to see its benefits; how they could kick ass even more at what they do Image: http://blog.crazyegg.com/lessons/show-benefits/ Make the personal benefits of being data-driven visible. Show how its autonomous pursuit enhances mastery. Super You! Data-drivenness You
  15. 15. Ownership is not about doing what I tell you to do is not about not doing what I tell you not to do is about doing what I don’t tell you to do Ensure ownership of benefits, not deliverables Focus on why, not what
  16. 16. ...Design Product Marketing Business Being data-driven requires much more than the right tools Organization supporting lean development Data-driven culture Key concepts Accessible tools Shared goals Management fostering self-direction Key concepts Key concepts
  17. 17. Data-drivenness, at both individual and team level, is a capability that can and should be developed http://skepdic.com/tencommands.html + Encourage autonomous reflection but ensure conceptual guidance Reflection Key concepts Being data-driven is not only about sensibly answering questions, but about finding them in the first place It cannot itself be standardized into rules or processes, but it can be guided by high-level key concepts
  18. 18. A core principle of learning is to be hypothesis-driven. Random experimentation will yield various outcomes but little insight Make continuous hypothesis-driven split testing the standard approach in all development activities across functions Hypothesis: [A statement that can be shown true or false] Design a test and a metric that confirms the hypothesis at a certain pre set limit Test in a real setting with real usersPay attention also to what is NOT happening Confirm or reject the hypothesis based on the chosen metric and pre set limit Add the learnings to your body of knowledge, also the invalidated hypotheses Build on the knowledge to form the next hypothesis
  19. 19. Data without insight can be destructive as it may lead development the wrong way look like improvement DAU (daily active users) Downloads (i.e. not even activations) Activity delta (change in user engagement) Churn (users who quit) > Vanity metric Indicator of actual use >Leading indicator of churn Actionable Lagging indicator Not actionable Put a lot of effort into designing KPIs (Key Performance Indicators) that are both sensible and actionable; such that steer development initiatives toward your ultimate business goal
  20. 20. Main KPIs should be static but reflect the current growth phase, while tactical tools can be whatever helps improve the KPIs High-level (growth-stage-specific) KPIs to continuously optimize Tactical tools do explore user behavior and to measure impact from change on KPI(s) http://lanesplitter.jalopnik.com/x-ray-artist-reveals-the-inner-beauty-of-classic-motorc-1698564514
  21. 21. ...Design Product Marketing Business Being data-driven requires much more than the right tools Organization supporting lean development Data-driven culture Key concepts Accessible tools Shared goals Management fostering self-direction Key concepts Key concepts
  22. 22. A simple tool that is actually used provides infinitely more value than a sophisticated one that is not https://xplr.com/xplr-umbrella-dataviz-on-top-of-unsupervised-machine-learning/ > Make sure everyone has easy access to all tools they might need and know how to use them
  23. 23. ...Design Product Marketing Business Being data-driven requires much more than the right tools Organization supporting lean development Data-driven culture Key concepts Accessible tools Shared goals Management fostering self-direction Key concepts Key concepts
  24. 24. Goals set at function level can often be unaligned or conflicting, breaking joint effort and leading to sub-optimal performance UX Set high level common goals to ensure everyone works and cooperates to optimize the whole as opposed to sub-optimizing their own components Business Manager goal: Visit eRPM (effective revenue per session) Product Manager goal: Avg. DAU (daily active users)
  25. 25. ...Design Product Marketing Business Being data-driven requires much more than the right tools Organization supporting lean development Data-driven culture Key concepts Accessible tools Shared goals Management fostering self-direction Key concepts Key concepts
  26. 26. ...Design Product Marketing Business Being data-driven requires much more than the right tools Organization supporting lean development Data-driven culture Key concepts Accessible tools Shared goals Management fostering self-direction Key concepts Key concepts
  27. 27. Example: Continuous improvement and A/B testing
  28. 28. A/B testing is about continuous and systematic gradual improvements that over time build increased performance 2.5x+ Improvement
  29. 29. Seemingly small but sustained improvements can have a very large impact on overall performance 49% Improvement min to max 25% Improvement mean to max
  30. 30. Example: Retention (how well you keep your users)
  31. 31. 6.1% Retention D1
  32. 32. Count of users by level
  33. 33. Count of users by level
  34. 34. Count of users by level
  35. 35. 6.1% Retention D1 47% Retention D1 25% Retention D7 6.2% Retention D90 Compare with 6.1% D1 Retention in the previous example
  36. 36. Example: LTV ((Customer) Life Time Value)
  37. 37. User profitability is basically determined by LTV* and CAC*. Being able to even roughly predict LTV can be very insightful. KPIs DAU Monthly rev. Retention D1 43 % Retention D7 24 % Retention D14 19 % Retention D30 12 % Conversion ARPDAU ARPPDAU LTV CAC ARPDAU nn € nn € Extr. tot. days (≤120th) nn nn LTV nn € nn € * LTV = (Customer) Life Time Value ; CAC = Customer Acquisition Cost
  38. 38. Example: Liigapörssi (KPIs and tools)
  39. 39. Direct Org. search Email SEM Display Other Our main time unit of comparable comparison is a game “phase”. The key desired user behavior varies within the phase F r e e Trades 1 Pool F r e e Trades 2 Pool F r e e Trades 6 Pool Direct Org. search Email SEM Display Other Pay (Revenue) Join pool (Activation) Sign up (Acquisition) Play the whole phase (Retention) Continue to next phase (Retention) Sign up or reactivate (Acquisition) Desired user behavior Create pool & invite (Referral) Invite friends (Referral) Make trades (Activation) Buy trades (Revenue) Create team (Acquisition) Phase 1 Phase 2 Phase 3-6 Season Acquisition Activation Retention Revenue Referral Acquisition Referral Revenue Retention Activation Focus area
  40. 40. We use various tools to identify development opportunities, run tests and measure impact on the main high-level KPIs set Area KPI Acquisition Signed up (%) Created team (%) CAC (€) Activation Active user (%) In pool (%) Retention In-phase retention (%) Phase change retention (%) Revenue LTV ARPPU Conversion Referral Invited friend (%) High-level (growth-stage-specific) KPIs to continuously optimize Tactical tools do explore user behavior and to measure impact from change on KPI(s) … … Funnels Emails Anything! Custom Dimensions using hashed user IDs enables user level analysis, which is VERY powerful
  41. 41. Some tools we are currently using Tool Domain Sought benefits Email Improve open rate and CTR Email (transactional) Identify issues in use and funnels Funnels Improve user onboarding and payments Ecommerce Research revenue by product, user type, etc. (Enh. Ecom. Trac.) Engagement Research anything at the user level! Engagement Retention, User level progression User Acquisition Volume and CPI (Cost Per Install) Landing pages Optimize CTA (Call To Action) conversion UI heat maps Research behavior and improve UI KPI dashboard Communication and awareness
  42. 42. Some suggested further reading * http://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-version Slideshare*
  43. 43. ...Design Product Marketing Business Being data-driven requires much more than the right tools Organization supporting lean development Data-driven culture Key concepts Accessible tools Shared goals Management fostering self-direction Key concepts Key concepts
  44. 44. @Hoverfalt
  • gabrieldo

    Apr. 11, 2018
  • LeonelFrattesi

    Dec. 3, 2017
  • RichardGustavsson1

    Oct. 25, 2017
  • powerirs

    Feb. 3, 2016
  • bardusco

    Feb. 3, 2016
  • IlkkaKalevi

    Oct. 13, 2015
  • TimoLehtinen2

    Sep. 22, 2015
  • TerhiHnninen

    Sep. 7, 2015

Video recording available here (YouTube): http://bit.ly/1KAmiCq

Vues

Nombre de vues

2 062

Sur Slideshare

0

À partir des intégrations

0

Nombre d'intégrations

68

Actions

Téléchargements

54

Partages

0

Commentaires

0

Mentions J'aime

8

×