In this talk, we will discuss automatic decision-making and AI techniques for customer relationship management. First, we will present a methodology that helps to develop highly automated promotion and loyalty management systems. Next, we will walk through practical examples of how advanced customer and content signals can be generated using predictive models, and how optimization and reinforcement learning techniques can be used for targeting, budgeting, and pricing decisions. This talk is for Data Scientists, Product Owners, and Software Engineers involved in marketing operations or development of marketing automation software and interested in ML-based decision automation techniques.
2. ML-based Decision Automation in Marketing Operations
● Billions of micro-decisions in real-time: who, when, how, what, ...
● Complex environment: human behavior, complex business models, hidden factors
● Many building blocks: propensity scoring, recommendation algorithms, multi-armed bandits, etc.
● How to design a system that can make micro-decisions based on business objectives?
4. Case Study: Decisions to be Automated
● Targeting – who
○ Exploits variability in tastes, price sensitivity, propensity to buy
○ Optimize short-term or long-term outcomes
● Timing – when
○ Exploits variability in price sensitivity
○ Exploits individual purchasing cycles
● Outreach/budgeting – how many
○ Exploits variability in propensity
● Promotion properties – what
○ Aggregated view on a promotion calendar
5. Approach
Retailers
Brands
Product
• Willingness to pay
• Stages of journey
• Affinities to brands
• Affinities to channels
Predictive Models
(Digital Twins)
• Propensity
• Life-time
value
• Demand
Economic Models
• What-if analysis
• Optimization
• Opportunity
finding
• Business
objectives
• Constraints
Controls
• Offers
• Channels
• Messages
• Prices
Signals Decisions
7. 7
Incremental revenue
Acquisition Maximization Retention
time
New Cardholder
$/brand
current non-buyers
+
high propensity to buy new product
current buyers
+
high propensity to buy more
current buyers
+
high propensity to buy less
Product Trial
Replenishment
Category Stretch
Retention Alarm
Com
petitive Defence
Look Alike Modeling and Survival Analysis
8. 8
Look Alike Modeling and Survival Analysis
time
no purchase
Model training
Model scoring
purchase
no purchase
behavioral history outcome
Customer
profiles for
training
Customer
profile to be
scored
score
10. Challenges with Basic Propensity Scoring
10
Checking
Account
Credit
Card
Brokerage
Account
Banking /
Telecom
Customer maturity
Product maturity level
Retail
● Does not take into account
product sequences
● Does not optimize offer
sequences (i.e. not strategic)
● Requires separate models
for different
products/offers/objectives
time
11. profile value (LTV / ROI)M
Offer 3
Offer 2
Offer 1
profile value (LTV / ROI)M
Offer 3
Offer 2
Offer 1
Next Best Action Model - Naive Approach
11
profile value (LTV / ROI)M
Time
Offer 1 Offer 2 Offer 3
Offer 3
Offer 2
Offer 1
12. Next Best Action with Reinforcement Learning
12
Customer state, t
action1
action2
action3
reward32
reward33
reward34
Customer state, t+1 Customer state, t+2 Customer state, t+3
Expected LTV / ROI
Q(s, a)
One
timer
Churner
Repeater
Loyal
customer
Multi
product
● Need to estimate an action-value
function given a certain offer policy:
State
(customer feature vector up to moment t)
Action
(offer feature vector)
● Use Q-function to optimize the offer
policy
s1
s2
s3
s4
s5
13. Next Best Action with Fitted Q Iteration (FQI)
13
Purchase
Visit
No action
Offer 1 Offer 2 Offer 3
2. Initialize approximate
repeat
1. Generate a batch of transitions
(each trajectory corresponds to 4 transitions):
{ (state, action, reward, new state) }
A simplified test dataset is shown for illustration
3. Initialize training set
4. For each
5. Learn new from training data
14. Next Best Action with FQI
14
Offer 3
Offer 2
Offer 1 (default)
Low state V
High state V
Customers who got
Offer 3 in early
Customers who got
Offer 2 early
Customers who got
Offer 2 -> Offer 3
Customers who did
not get offers or got
Offer 1
● Max value for each state:
● Next best action for each state (policy):
A simplified test dataset is shown for illustration
15. Next Best Action with FQI
15
● A generalization of the look alike modeling for multi-step and/or multi-choice strategies
● More control over LTV/ROI metrics
● Can evaluate performance of a new policy based on historical trajectories
● Batch-online learning trade-off: multi armed bandits
19. 19
Campaign Parameters Optimization
Purchase
trigger
buy <X buy X+
buy 0 buy 1+
Announcement
Buy X or more units
and save on your
next shopping trip!
Promotion
Y% off
1. Estimate demand elasticity
2. Estimate how many
consumers will buy more,
how many will redeem offers
3. Do break-even analysis for
costs and benefits
21. Objective Selection
Plan and Forecast
Review
User Experience
Execution and
Measurement
Privileged and Confidential 21
Solution Design: Marketer’s Perspective