Talk at 2nd Workshop on New Trends in Content-Based Recommender Systems, RecSys 2015 Vienna #recsys2015 #CoCoS Authors: Lucas Bernardi, Jaap Kamps, Julia Kiseleva and Melanie Mueller. Booking.com, Amsterdam University, Eindhoven University.

1. Booking.com
The Continuous
Cold Start Problem
in e-Commerce
Recommender Systems
RecSys Vienna 2015-09-20
Lucas Bernardi, Melanie Mueller
, Amsterdam
Jaap Kamps, Julia Kiseleva
Amsterdam & Eindhoven University

2. Booking.com
Cold Start in Travel

3. Booking.com
Cold Start in Travel

4. Booking.com
Travel recommendations
Destinations related to Vienna:
• User-to-user collaborative filtering:
• Content-based item-to-item recommendation:
Customers who viewed Hotel Sacher Wien also viewed:

5. Booking.com
• Characteristics of the
Continuous Cold Start Problem
Outline
• Addressing the
Continuous Cold Start Problem

6. Booking.com
Recommender Systems
5 1
?
? ?
2
5
4
3
• Task: predict rating of new item
• Classical cold start: too many ?
???
new
user
?
?
?
?
new item

7. Booking.com
• Cold start = not enough information (yet)
Classical Cold Start
→ Bridge period until warmed up
Other information:
popularity, content...
Standard
recommender
time →
performance

8. Booking.com
• Classical cold-start / sparsity
new / rare users
User Continuous Cold Start
• Volatility
• Personas
• Identity

9. Booking.com
• New users:
- Millions unique users/day, growing
- Most not logged-in, no cookie
Cold Users
1 51 101 151 201 251 301 351
Day
ActivityLevel
Continuously cold users at Booking.com. Activity levels of two randomly chosen users
ver time. The top user exhibits only rare activity throughout a year, and the bottom use
• Sparse users: holidays are rare events!

10. Booking.com
• Classical cold-start / sparsity
new / rare users
User Continuous Cold Start
• Volatility
user interest changes over time
• Personas
• Identity

11. Booking.com
Backpacker hostel
User Volatility
Family hotel
Wellness hotel

12. Booking.com
• Classical cold-start / sparsity
new / rare users
User Continuous Cold Start
• Volatility
user interest changes over time
• Personas
different interest at close-by points in time
• Identity

13. Booking.com
• Leisure versus business:
User Personas
1 11 21 31 41 51 61 71 81 91
Day
ActivtyLevel
Leisure
booking
Business
booking
inuously cold users at Booking.com. Activity levels of two randomly chosen
me. The top user exhibits only rare activity throughout a year, and the botto
nas, making a leisure and a business booking, without much activity inbetween
• Browsing on a sunny versus rainy day
• Weekend city trip versus long holiday

14. Booking.com
• Classical cold-start / sparsity
new / rare users
User Continuous Cold Start
• Volatility
user interest changes over time
• Personas
different interest at close-by points in time
• Identity
failure to match data from same user

15. Booking.com
User Identity Problem
• Different devices
• Not logged in

16. Booking.com
• Classical cold-start / sparsity
new / rare items
Item Continuous Cold Start
• Volatility
item properties/values change over time
• Personas
item appeals to different types of users
• Identity
failure to match data from same item

17. Booking.com
• Characteristics of the
Continuous Cold Start Problem
Outline
• Addressing the
Continuous Cold Start Problem

18. Booking.com
Addressing Continuous Cold Start
- Ask user
• Continuous cold start = information continuously missing
→ Need more/other information:

19. Booking.com
Ask the User

20. Booking.com
- Implicit ratings:
buys, clicks, views...
Addressing Continuous Cold Start
- Ask user
• Continuous cold start = information continuously missing
→ Need more/other information:
→ can add friction

21. Booking.com
Implicit Ratings
Customers who viewed Hotel Sacher Wien also viewed:

22. Booking.com
- Implicit ratings:
buys, clicks, views...
Addressing Continuous Cold Start
- Ask user
• Continuous cold start = information continuously missing
→ Need more/other information:
- Popularity
→ can add friction
→ weak signal

23. Booking.com
Destination Finder

24. Booking.com
- Implicit ratings:
buys, clicks, views...
Addressing Continuous Cold Start
- Content
- Ask user
→ surprisingly hard to beat!
→ not personalized
- item descriptions
- user profiles
- context
• Continuous cold start = information continuously missing
→ Need more/other information:
- Popularity
→ can add friction
→ weak signal

25. Booking.com
Destination Finder

26. Booking.com
Destination Finder

27. Booking.com
Endorsements
• From users who stayed at hotel in destination
• Free text endorsements since 2013
• 2014: used NLP techniques to extract 256 tags
• 13 000 unique endorsements
• 60 000 destinations

28. Booking.com
Endorsements

29. Booking.com
Endorsements

30. Booking.com
Destination Finder Cold Start
• Upper funnel: almost no information about user (very cold)
• Construct MIXED profile:
Features:
- user: operating system, browser
- context: weekday
- item rating: endorsement
<Ubuntu, Firefox, Tuesday, opera>
<0,1,0,0,0,1,0,0,1,0,0,0,0,0,0,0,1,0...>
• Train recommender for each profile
• New user + query: find closest profile
• Clustering → mixed content profiles

31. Booking.com
A/B testing
Recommender Users Click-Through-Rate
No mixed profiles 13,306 18.5 ± 0.4%
With mixed profiles 13,562 22.2 ± 0.4%
Improved by 20%

32. Booking.com
- Implicit ratings:
buys, clicks, views...
Addressing Continuous Cold Start
- Content
- Ask user
→ surprisingly hard to beat!
→ not personalized
- item descriptions
- user profiles
- context
• Continuous cold start = information continuously missing
→ Need more/other information:
- Popularity
→ can add friction
→ weak signal
→ mix ‘em up!
Not only bridge warm-up period,
but deal with continuous cold start

33. Booking.com
• Classical cold-start
new & rare users / items
Summary: Continuous Cold Start
• Volatility
users/items change over time
• Personas
different interest at close-by times
• Identity
failure to match data from same user/item

34. Booking.com
Timely Personalized Recommendation:
Don’t do lunch cold start