Games earn more money than movies and music combined. That means a lot of data is generated as well. One of the development considerations for ML Pipeline is that it must be easy to use, maintain, and integrate. However, it doesn’t necessarily have to be developed from scratch. By using well-known libraries/frameworks and choice of efficient tools whenever possible, we can avoid “reinventing the wheel”, making it flexible and extensible.

1. Using Machine Learning at
Scale: A Gaming Industry
Experience!
Arthur Gola
Data Science Manager @ Wildlife Studios
Vini Jaiswal
Customer Success Engineer @ Databricks

2. Agenda
01
02
03
04
05
2
Gaming Industry Trends
Data Science Platform @ Wildlife
Use Case: Personalized Offers
Other Use Cases @ Wildlife

3. Gaming
industry
trends
01

4. Mobile is the largest
and fastest-growing
platform within video
games
Market
Global video game market:
$159B in 2020
Out of which $77Bin mobile
(that's more than music and movies markets combined)
Mobile grows 13.3%p.a.
(vs. to 5% for PC and 7% for console)
Source: Newzoo 2020 Global Games Market Report

5. We develop games that make
billions of players happier.
Wildlife is a blend of excellence in art and creativity with data science and high-end technology

6. Top 10 largest mobile gaming
companies in the world
2.5 billion downloads to date
10s of terabytes of new data
every day
petabytes processed daily in
core pipelines
60+ games released
Wildlife in numbers

7. Data Science
Platform at
Wildlife
02

8. Languages
Tracking Projects Models
Massivelyscalabledatacleansing&transformation
End to end ML
lifecycle
ETL / Data
Processing
Data Science Platform
APIs
Jobs
Models
Notebooks
Dashboards
ML Runtime
Databricks
Runtime
BRONZE
SILVER
GOLD
DELTA
Metastore
BI/Dashboarding
In-house
solution
Marketing Analysts
Product Managers
Wildlife's unified data platform
Wildlife Games
Credits:
Pedro Sereno Galvão
kafka
connect

9. Case:
Personalized
Offers
03

10. The magic of finding the right product for the
right user
● Over 60% of our revenue comes from in-app purchases — and offers
are a large portion of that
● Users are very different — 95% of our user base never make a
purchase
● From those that do make a purchase, only 5% are responsible for
more than half the revenue
● Hardcore vs casual users
Zooba
War Machines
Tennis Clash

11. We started simple and grew in complexity
Frequency: the best frequency for the game
Price: the right offer size for each wallet
Discount: higher discounts may increase cannibalization
Content: dimensionality explodes — how can we simplify?

12. We started simple and grew in complexity
Frequency: the best frequency for the game
Price: the right offer size for each wallet
Discount: higher discounts may increase cannibalization
Content: dimensionality explodes — how can we simplify?

13. We started simple and grew in complexity
Frequency: the best frequency for the game
Price: the right offer size for each wallet
Discount: higher discounts may increase cannibalization
Content: dimensionality explodes — how can we simplify?

14. We started simple and grew in complexity
Frequency: the best frequency for the game
Price: the right offer size for each wallet
Discount: higher discounts may increase cannibalization
Content: dimensionality explodes — how can we simplify?

15. Offer Personalization System Overview
Backend Servers
(Go)
Offers Gateway
(Go)
Model API
(python)
A.pkl
Model API
(python)
B.pkl
Experimentation
Platform
(Go)
Feature Store
(DynamoDB)
Analytics
(In-house solution + Looker)
Data Lake
(S3)

16. Backend Servers
(Go)
Offers Gateway
(Go)
Model API
(python)
Model API
(python)
Experimentation
Platform
(Go)
Feature Store
(DynamoDB)
Analytics
(In-house solution + Looker)
Data Lake
(S3)
A.pkl
B.pkl
1. The game
requests an offer
Offer Personalization System Overview

17. Backend Servers
(Go)
Offers Gateway
(Go)
Model API
(python)
Model API
(python)
Experimentation
Platform
(Go)
Feature Store
(DynamoDB)
Analytics
(In-house solution + Looker)
Data Lake
(S3)
A.pkl
B.pkl
2. The backend servers
forward that request to
the Offers Gateway
Offer Personalization System Overview

18. Backend Servers
(Go)
Offers Gateway
(Go)
Model API
(python)
Model API
(python)
Experimentation
Platform
(Go)
Feature Store
(DynamoDB)
Analytics
(In-house solution + Looker)
Data Lake
(S3)
A.pkl
B.pkl
3. Offers Gateway
requests the
experimentation config
Offer Personalization System Overview

19. Backend Servers
(Go)
Offers Gateway
(Go)
Model API
(python)
Model API
(python)
Experimentation
Platform
(Go)
Feature Store
(DynamoDB)
Analytics
(In-house solution + Looker)
Data Lake
(S3)
A.pkl
B.pkl
4. Experimentation
platform returns the
test config
Offer Personalization System Overview

20. Backend Servers
(Go)
Offers Gateway
(Go)
Model API
(python)
Model API
(python)
Experimentation
Platform
(Go)
Feature Store
(DynamoDB)
Analytics
(In-house solution + Looker)
Data Lake
(S3)
A.pkl
B.pkl
5. Gateway sends
request to the
appropriate Model API
Offer Personalization System Overview

21. Backend Servers
(Go)
Offers Gateway
(Go)
Model API
(python)
Model API
(python)
Experimentation
Platform
(Go)
Feature Store
(DynamoDB)
Analytics
(In-house solution + Looker)
Data Lake
(S3)
A.pkl
B.pkl
6. Model API requests
the user's features from
a Feature Store
Offer Personalization System Overview

22. Backend Servers
(Go)
Offers Gateway
(Go)
Model API
(python)
Model API
(python)
Experimentation
Platform
(Go)
Feature Store
(DynamoDB)
Analytics
(In-house solution + Looker)
Data Lake
(S3)
A.pkl
B.pkl
7. Feature Store
responds with features
from the user
Offer Personalization System Overview

23. Backend Servers
(Go)
Offers Gateway
(Go)
Model API
(python)
Model API
(python)
Experimentation
Platform
(Go)
Feature Store
(DynamoDB)
Analytics
(In-house solution + Looker)
Data Lake
(S3)
A.pkl
B.pkl
8. Model API generates
the recommendation
Offer Personalization System Overview

24. Backend Servers
(Go)
Offers Gateway
(Go)
Model API
(python)
Model API
(python)
Experimentation
Platform
(Go)
Feature Store
(DynamoDB)
Analytics
(In-house solution + Looker)
Data Lake
(S3)
A.pkl
B.pkl
9. Gateway forwards
recommendation to
backend servers
Offer Personalization System Overview

25. Backend Servers
(Go)
Offers Gateway
(Go)
Model API
(python)
Model API
(python)
Experimentation
Platform
(Go)
Feature Store
(DynamoDB)
Analytics
(In-house solution + Looker)
Data Lake
(S3)
A.pkl
B.pkl
10. Offer is shown to
the user
Offer Personalization System Overview

26. Backend Servers
(Go)
Offers Gateway
(Go)
Model API
(python)
Model API
(python)
Experimentation
Platform
(Go)
Feature Store
(DynamoDB)
Analytics
(In-house solution + Looker)
Data Lake
(S3)
A.pkl
B.pkl
During the process, all
components are sending data to
our storage that will feed into our
monitoring system
Offer Personalization System Overview

27. Wildlife's ML library speeds up model training
Application-agnostic
MLFlow-integrated
Runs locally or on
Databricks clusters
Standardized training and
deployment
Automated reporting: business
metrics, model error, feature
importance
Hyperparameter optimization

28. Example: training notebook walkthrough
Step zero: Data Collection process creates the dataset (will not be covered)
1. Installation
2. Game-specific parameters
a. Load dataset
b. Define mapping of model output to server payload
3. Model training: the compound classifier
a. Define training parameters
b. Execute training
4. Test and evaluation
5. Deployment

29. Example: training notebook walkthrough

30. It's not just cool, it's profitable!

31. So, what is next? Maybe Reinforcement Learning.
We should find the sequence of offers that maximizes long-term rewards
We should have a retraining framework that takes into account the explore-exploit tradeoff
We should explore our search space wisely — too many dimensions, especially with content
Yes, that is right, we are testing a Reinforcement Learning framework — stay tuned for
news on that!

32. Other use
cases
04

33. There is much more to Data Science at Wildlife!
● Rating and matchmaking systems that predict user performance and create fair
matchups for fun and engaging games
● Advanced experimentation techniques such as bayesian optimization, multi-armed and
contextual bandits to test more hypotheses, learn and improve faster
● LTV models that allow for a optimized, highly profitable marketing operation
● Ads monetization system that continuously maximizes ads revenues

34. Thank you!
www.wildlifestudios.com
arthur.gola@wildlifestudios.com

35. Feedback
Your feedback is important to us.
Don’t forget to rate
and review the sessions.