Importance of Data Driven Approach to Manage Digital Businesses1. © Play Games24x7 Pvt. Ltd.
Importance of data driven approach to
manage digital businesses
12th January, 2015
Sachin Uppal, Marketing Director
Play Games24x7 Pvt. Ltd.
www.RummyCircle.com
2. © Play Games24x7 Pvt. Ltd.
“You can’t manage what you don’t
measure.”
W. Edwards Deming and Peter Drucker
3. © Play Games24x7 Pvt. Ltd.
1. Data driven online businesses
i. Online business definition
ii. Data driven approach
iii. Key online business components
2. Culture of Analytics
i. Why the data driven approach?
ii. Business intelligence structure
iii. Optimization, Behavioral Learning, Big data & Predictive Analytics
3. Importance of Measurement – Case Studies
i. Acquisition – Consumer Behavior and Preferences
ii. Product – Design, Pricing and Revenue optimization
iii. Retention – Big data driven predictive analytics
iv. Risk Management - Process Automation
4. Future Trends
Agenda
4. © Play Games24x7 Pvt. Ltd.
1. Data driven online businesses
i. Online business definition
ii. Data driven approach
iii. Key online business components
2. Culture of Analytics
i. Why the data driven approach?
ii. Business intelligence structure
iii. Optimization, Behavioral Learning, Big data & Predictive Analytics
3. Importance of Measurement – Case Studies
i. Acquisition – Consumer Behavior and Preferences
ii. Product – Design, Pricing and Revenue optimization
iii. Retention – Big data driven predictive analytics
iv. Risk Management - Process Automation
4. Future Trends
Agenda
5. © Play Games24x7 Pvt. Ltd.
A business or Non-Profit organization that sells its products and services on any
digital delivery platform (like: Website, Mobile, Tablet, Digital TV etc.) where
product / service content is delivered through internet.
• Case in point, Play Games24x7’s Online Rummy game business,
RummyCircle.com, where the service is delivered online.
P.S.: I have limited the definition to a fairly tactical one to keep things in perspective. I am not covering, corporate social
responsibility, impact on environment, impact on lives of people and world peace at this moment.
1.1 Online Business Definition
6. © Play Games24x7 Pvt. Ltd.
Case in point – RummyCircle.com
RummyCircle: India’s
best and largest online
rummy site
3 offices, 160
employees
Funded by Tiger GlobalCash positive and
growing!
We are hiring!
10. © Play Games24x7 Pvt. Ltd.
1.2 Data Driven Approach
• Business Objective: Based on our strategy, we set a business objective. This is also based on
past data of what works and what doesn’t.
• Analysis: We break down the business problem by looking at the relevant data.
• Hypothesis: Basis our analysis, we come up with Hypothesis that could explain the behavior.
• Experiment: We setup experiments to test our hypothesis.
• Outcome: We measure the results and report our outcomes.
• Insight: We learn from the outcomes.
This process is repeated and we continuously improve our business decisions and outcomes.
11. © Play Games24x7 Pvt. Ltd.
Key business components from growth perspective:
• New User Acquisition - New players through attractive campaigns
• Product Design - Provide a wholesome Rummy experience
• Payments – Seamless multi-option payment mechanism
• Retention - Innovative offers and promotions
Focus on Customer service components:
• Customer Delight – Low Turn-Around-Times
• Smooth Operations – Time bound delivery / on -schedule
• Risk Management – Safe, fair, low risk - enhance consumer trust
1.3 Key Online Business Components
12. © Play Games24x7 Pvt. Ltd.
1. Data driven online businesses
i. Online business definition
ii. Data driven approach
iii. Key online business components
2. Culture of Analytics
i. Why the data driven approach?
ii. Business intelligence structure
iii. Optimization, Behavioral Learning, Big data & Predictive Analytics
3. Importance of Measurement – Case Studies
i. Acquisition – Consumer Behavior and Preferences
ii. Product – Design, Pricing and Revenue optimization
iii. Retention – Big data driven predictive analytics
iv. Risk Management - Process Automation
4. Future Trends
Agenda
13. © Play Games24x7 Pvt. Ltd.
Business
Intelligence
Engineering
Acquisition
Risk
Management
Operations
Customer
Support
Retention
Product
2. Culture of Analytics – Across Business
Data exchange
• Data helps in making Fact /
Evidence based decisions.
Faster and more accurately.
• It’s no more an activity or
department, it is the Culture.
15. © Play Games24x7 Pvt. Ltd.
Business
Intelligence
Engineering
Acquisition
Risk
Management
Operations
Customer
Support
Retention
Product
2.1 Why Data driven approach?
Data exchange
Facebook Ad 1
or Ad 2? Green
or Red colour?
Limit Rs. 1000
or Rs. 5000?
Tournament at 9
PM or 10 PM?
I responded
within 2 hours!
How much bonus
to whom? Rs. 1000
or Rs.200?
2 Steps or 3 Steps?
4 options or 5?
Add 2 or 3
servers? Tech. 1
or 2?
17. © Play Games24x7 Pvt. Ltd.
2.2 Tools and Techniques
Below are some standard tools and techniques we employ on an ongoing basis:
• Behavior or Preference measurement - A/B Testing
• Look for Statistical Significance – for complete funnel
• Monte-Carlo Simulation (smaller data samples)
• E.g.: Acquisition and Product tests.
• Regression Analysis
• Predicting Y with variables X1, X2, X3… Xn
• E.g.: Display advertising has an impact on brand search volume.
• Big Data
• Predictive models based on real-time events and / or past data for real time actions or predictive outcomes.
E.g.: Retention case study.
Image Source
18. © Play Games24x7 Pvt. Ltd.
2.3 Business Intelligence Structure
BI enables the business to make intelligent, fact-based decisions
20. © Play Games24x7 Pvt. Ltd.
1. Data driven online businesses
i. Online business definition
ii. Data driven approach
iii. Key online business components
2. Culture of Analytics
i. Why the data driven approach?
ii. Business intelligence structure
iii. Optimization, Behavioral Learning, Big data & Predictive Analytics
3. Importance of Measurement – Case Studies
i. Acquisition – Consumer Behavior and Preferences
ii. Product – Design, Pricing and Revenue optimization
iii. Retention – Big data driven predictive analytics
iv. Risk Management - Process Automation
4. Future Trends
Agenda
21. © Play Games24x7 Pvt. Ltd.
1. Acquisition – Consumer Behavior and
Preferences
2. Product – Design, Pricing and Revenue
optimization
3. Retention – Big data driven predictive
analytics
4. Risk Management – Process Automation
Importance of Data Driven
Approach - 4 Case Studies
22. © Play Games24x7 Pvt. Ltd.
i. Acquisition – Consumer Behavior and Preferences
ii. Product – Design, Pricing and Revenue optimization
iii. Retention – Big data driven predictive analytics
iv. Risk Management – Case for automation
Four Real Life Situations
23. © Play Games24x7 Pvt. Ltd.
Challenge: Cost per acquisition (CPA) grows significantly when we scale-up the new
user acquisitions.
Objective: Scale-up new users acquisitions while keeping CPAs low.
Approach: Full funnel optimization through multiple experiments.
Acquisitions – A scale-up challenge
24. © Play Games24x7 Pvt. Ltd.
Approach: Full funnel optimization (FFO) through multiple experiments.
• Optimize Media buys – Reduce Cost
• Optimize Creatives – Improve Conversions – Reduce Cost / Lead
• Optimize Landing Pages – Improve Conversions – Reduce CPA
Acquisitions – A scale-up challenge
25. © Play Games24x7 Pvt. Ltd.
Approach: Full funnel optimization (FFO) through multiple experiments.
• Optimize Media buys – Reduce Cost
• Optimize Creatives – Improve Conversions – Reduce Cost / Lead
• Optimize Landing Pages – Improve Conversions – Reduce CPA
Acquisitions – A scale-up challenge
26. © Play Games24x7 Pvt. Ltd.
Business Objective: Bring down CPA
Hypothesis: Aggregated demographic targeting increases CPA.
Experiment: Micro-segmentation of demographic to identify the high converting
audience
• Data helped us identify the highly converting demographic
• We made micro-segments of that audience
• Highly focused media buys to scale-up campaign
Outcome: CPA down by 10%
Insight: Micro-segmentation leads to more structured spends and lowers CPA.
Full Funnel Optimization – Media Buys
27. © Play Games24x7 Pvt. Ltd.
Approach: Full funnel optimization (FFO) through multiple experiments.
• Optimize Media buys – Reduce Cost
• Optimize Creative – Improve Conversions – Reduce Cost / Lead
• Experiment 1: Simplicity Vs Complexity
• Optimize Landing Pages – Improve Conversions – Reduce CPA
FFO - Creatives
28. © Play Games24x7 Pvt. Ltd.
Simple Moderately Complex
FFO – Experiment
Which one won?
29. © Play Games24x7 Pvt. Ltd.
Simple Moderately Complex
FFO – Experiment
Moderately complex creative showed an 18% Improvement in Visit to Paid rate.
30. © Play Games24x7 Pvt. Ltd.
Business Objective: Reduce CPA by improving conversions.
Hypothesis: Absolute simplicity is better in interactive banners.
Experiment: We made two versions of an ad to test if Absolutely Simplicity in
Interactive Banner works better than Moderate Complexity.
Outcome: While Absolute Simplicity drives higher clicks, it doesn’t drive higher
conversions as audience need a problem that they think is challenging but quickly
solvable.
Insight: Players need a challenge that is solvable, quickly.
FFO – Experiment
31. © Play Games24x7 Pvt. Ltd.
Approach: Full funnel optimization (FFO) through multiple experiments.
• Optimize Media buys – Reduce Cost
• Optimize Creative – Improve Conversions – Reduce Cost / Lead
• Optimize Landing Pages – Improve Conversions – Reduce CPA
FFO – Landing Pages
33. © Play Games24x7 Pvt. Ltd.
Business Objective: Increase Click to Visit Rate and Visit to Lead Rates
Hypothesis: Registration form has multiple elements that need to be optimized.
Experiments: Alter the registration form components like:
• Number of fields
• Left or Right
• Register for Free
• Bonus message
Outcome: CTV up by 25%. LTP up by 20%.
FFO - Landing Pages
34. © Play Games24x7 Pvt. Ltd.
i. Acquisition – Consumer Behavior and Preferences
ii. Product – Design, Pricing and Revenue optimization
iii. Retention – Big data driven predictive analytics
iv. Risk Management – Case for automation
Four Real Life Situations
37. © Play Games24x7 Pvt. Ltd.
Product – Shopping cart challenge
Challenge: Flat-lining conversion funnel percentages
Approach: Optimize Shopping cart (or Add Cash funnel) funnel for conversions
through multiple experiments:
• Build plug-n-play framework for running A/B tests
• First experiment - Reduce the number of choices
38. © Play Games24x7 Pvt. Ltd.
Funnel Optimization - Reduced Choices
Objective: Increase conversion rate.
Hypotheses: More choice in selecting amounts results in lower conversion.
Experiment: Split users entering the funnel into two groups randomly
• Control group - 5 amount tiles
• Test group - 4 amount tiles
41. © Play Games24x7 Pvt. Ltd.
Outcome:
• Increase in conversions by 12% in the reduced options path
• Increase in average deposit amount by 15%
• Increase in ARPU by 17%
Insights:
• Less is more – helps players make a quick decision
• Behaviour – Propensity to deposit higher amounts reduces when lower amounts
are available
Experiment Results
42. © Play Games24x7 Pvt. Ltd.
i. Acquisition – Consumer Behavior and Preferences
ii. Product – Design, Pricing and Revenue optimization
iii. Retention – Big data driven predictive analytics
iv. Risk Management – Case for automation
Four Real Life Situations
43. © Play Games24x7 Pvt. Ltd.
Retention - High Early Churn Rates
Challenge: A significant proportion of players do not last beyond first day on the
site.
Objective: To improve early retention of new cash players
Approach: Offer a strong incentive to players sitting on the fence to return to play.
44. © Play Games24x7 Pvt. Ltd.
Context – Churn Trends
Problem: How do you substantially improve the Day1 retention while ensuring a
positive net return?
Points to consider:
• The incentive does not affect players who are likely to return anyway.
• The churn patterns beyond the first day are not expected to change.
How do you make an attractive yet profitable offer?
45. © Play Games24x7 Pvt. Ltd.
Identify which players are likely to return and which players are potentially one-
timers.
Using this model, we performed an a priori segmentation of players into Fence-
Sitters and Enthusiasts.
The Proposed Solution – Segment Identification
48. © Play Games24x7 Pvt. Ltd.
Insight: Customization of incentives and journeys for different segments can
significantly increase the impact of any promotion.
Results
Category Day1 Retention
Fence-Sitters: Test 59%
Fence-Sitters: Control 54%
Enthusiasts 67%
Over a 2-month period, this campaign yielded an ROI of 62%.
49. © Play Games24x7 Pvt. Ltd.
i. Acquisition – Consumer Behavior and Preferences
ii. Product – Design, Pricing and Revenue optimization
iii. Retention – Big data driven predictive analytics
iv. Risk Management – Case for automation
Four Real Life Situations
50. © Play Games24x7 Pvt. Ltd.
Risk Management – KYC verification for high value players
Background
• For a real money game company its very important that players play within limits.
• We give a small limit to players when they join the site.
• To protect players’ and manage risk, we follow a KYC process before increasing limits.
Challenge: KYC process is slow and people don’t want to share identity documents.
Objective: To increase KYC verified players without a monetary incentive.
Hypothesis: Offering a faster service as an incentive will increase KYC verification.
51. © Play Games24x7 Pvt. Ltd.
Risk Management – KYC verification for high value players
Experiment 1: This was a Time Series experiment. We started offering 1 Day withdrawal
processing time to KYC verified players.
Offer:
Results: Avg. daily KYC verified members increased by 200% and it sustained.
Experiment 2: This was a Time Series experiment too. To increase the numbers further, we
automated the document uploading process.
Automation: Once a player is eligible, show a message to players on website and ask them to
upload KYC documents instead of calling and asking them to upload documents.
Results: KYC verified players are now 1200% higher / day and it’s sustaining.
Withdrawal Processing Time
KYC Verified 1 Day
Not Verified 2 Days
52. © Play Games24x7 Pvt. Ltd.
• High value players added more cash thus leading to increase in cash deposits and
revenue.
• Taking actions to save people’s time is valuable and leads to revenue growth.
• Automation of processes that has significant impact on saving people’s time
should be taken up with priority for scaling up.
Insights
53. © Play Games24x7 Pvt. Ltd.
1. Data driven online businesses
i. Online business definition
ii. Data driven approach
iii. Key online business components
2. Culture of Analytics
i. Why the data driven approach?
ii. Business intelligence structure
iii. Optimization, Behavioral Learning, Big data & Predictive Analytics
3. Importance of Measurement – Case Studies
i. Acquisition – Consumer Behavior and Preferences
ii. Product – Design, Pricing and Revenue optimization
iii. Retention – Big data driven predictive analytics
iv. Risk Management - Process Automation
4. Future Trends
Agenda
54. © Play Games24x7 Pvt. Ltd.
1) Multi-platform product / service
• Desktop, Laptop, Mobile, Tablet
2) Social Gaming
Future Trends
55. © Play Games24x7 Pvt. Ltd.
Rummycircle.com – Mobile & Beyond
• We estimate a potential market size of 1 million active rummy players and hence
expect our rummy business to grow rapidly for the next several years.
• We expect mobile to be a big driver of our growth.
• Smart phone sales growing close to 230% year on year.
• Smart phone users are expected to hit the 150 million mark by the end of 2015.
• The active 3G base is expected to hit the 100 million mark by the end of 2015.
• We released Rummycircle on mobiles and tablets in April 2014.
• In November, 12% of our new cash players joined us from their mobile devices.
• Over 10% of our revenue is now generated on our mobile platform.
56. © Play Games24x7 Pvt. Ltd.
To date, we are the only company in India to offer cash games on
mobiles and tablets.
57. © Play Games24x7 Pvt. Ltd.
• The current $10 billion casual gaming (social + mobile) market is expected to
double to $20 billion by 2017.
Social Gaming
• To tap into this massive opportunity
we launched Call It Right, our first
social game internationally on
Facebook, Android, iOS and Yahoo.
58. © Play Games24x7 Pvt. Ltd.
• Given the rapid proliferation of smart phones in India, we foresee the mobile
gaming space in India to take off in a very big way.
• We expect to launch our first game in the Indian market in Q1 2015 and have a
whole suite of games lined up for release after that in 2015 alone.
Social Gaming in the Indian Market
59. © Play Games24x7 Pvt. Ltd.
Connect on Twitter: @sachinuppal
LinkedIn: in.linkedin.com/in/sachinu/
Email me: Sachinuppal AT Gmail.com
Questions
Notes de l'éditeur I will be covering Importance of data driven approach to manage digital businesses through real life case-studies.
I am Sachin Uppal. Marketing Director at Play Games24x7 Pvt. Ltd. And we own and operate www.RummyCircle.com, India’s largest online rummy website.
If you are an entrepreneur, startup CEO, aspiring to startup, marketer, online marketer, digital marketer, data analyst, data scientist, marketing analyst, risk management, big data, predictive analytics and have made data driven decisions or want to learn how to make data driven decisions, this presentation will be helpful. You can’t manage what you don’t measure.
It’s a very popular statement and has been credited to American Engineer and Statistician W. Edwards Deming and Australian born management consultant Peter Drucker.
This is of extreme importance to us in our organization and lives. I will be discussing online business keeping in mind this definition.
Specifically, I will be covering RummyCircle.com under this definition and how we operate.
We run a very profitable online games business.
Our service delivery is a Digital delivery – No physical goods.
We host rummy games for cash and charge a fee for every cash rummy game.
Our growth is hence an outcome of how we scale cash rummy games. Our rapid growth and focus on delivering nothing but the Best Rummy Experience has made us the market leaders in Rummy.
We became cash flow positive in 18 months of our operations and have growth 10 X in past 6 years.
Our impressive growth surely made some leading investment houses fund our growth initiatives. We presently have 3 offices and we are hiring! Our phenomenal growth in the past 5-6 years is an outcome of “How we apply data driven approach to make to our everyday decisions”.
Play time in minutes is a good indication of how we might be making revenues. While many organizations talk about being data driven approach, “how” you run your experiments matters and how you use that data to make decisions matters a lot. We analyze data from existing user behavior and their actions on the site and then come up with hypothesis on why something is happening or how we can improve on what is happening and test it out by creating new features and running experiments and measuring data and analyzing the same and proving or disproving our hypothesis. We analyze data from existing user behavior and their actions on the site and then come up with hypothesis on why something is happening or how we can improve on what is happening and test it out by creating new features and running experiments and measuring data and analyzing the same and proving or disproving our hypothesis. We track and measure almost everything across departments so that we can manage and optimize.
Makes the decision making process robust with Evidence based decisions.
Makes it easy to Access and Share Information across the board.
Enables real time analysis for faster turn around.
Helps in identifying the wasteful activities in the business.
Reduces the business risks and bottlenecks.
Off-course the outcome is business growth.
How a data driven approach as a culture helps in decision making across the company. The tools and techniques we employ have become part of the famous comic character Dilbert. A/B testing is one of the most extensively used Technique at RummyCircle. I am now going to share real life problems that we saw in our business and how we went about solving these in a case-study format.
The 4 cases or business problems I am going to talk about is Acquisition, Product design, Retention and Risk Management
In the Acquisition case study, I am going to cover, how we learn about consumer preferences and un-cover consumer behaviour patters. And how this leads to higher conversions.
In the Product case study, I am going to cover, how we arrived at a pricing decision and increased revenues through product design experiments.
In the customer Retention case study, I will cover, how we used Big Data driven predict modelling to optimize costs and improve revenues.
In the Risk management case study, I will cover, how we built a case for automation and saw massive improvements in time and cost savings. We saw a remarkable response to this initiative as the average number of successful KYC verifications grew by over 200%
It was a win-win situation for all stakeholders as we better protected ourselves by verifying these high value players as per KYC guidelines while with higher validation level, these players enjoyed higher play limits leading to increase in the revenue.