Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Exploration and diversity in recommender systems
Similar to Exploration and diversity in recommender systems (20)
Recently uploaded
Recently uploaded (20)
Exploration and diversity in recommender systems
- 1. © Tubi, proprietary and confidential Exploration and Diversity in Recommender Systems Jaya Kawale Director of Machine Learning 2020/08/13
- 2. © Tubi, proprietary and confidential 2 Tubi ● AVOD (Advertising based Video On Demand) ● Acquired by Fox. ● Surpassed 200M hours viewed per month in June.
- 3. © Tubi, proprietary and confidential 3 ML at Tubi Important areas ● Personalization ML ● Content ML ● Ads ML
- 4. ML for Personalization ● Homepage recommendations ● Container/rows ordering ● Search ● Notifications
- 5. 1st & 3rd Party Data Audience Assessment Viewer-oriented data Universe of Content + Metadata Title-orienteddata Products Models Embeddings (CTXT, MD, MMD, Genre, Demos, Actor, et al) Use Cases Beam from Universe to Tubiverse Cold➔ Warm➔Hot Starting Content Value Assessment Tiering Inventory in Tubiverse Augmented Search Seeding Growth Coordinated Pursuit of New Audience Portfolio Analysis / Simulation ML for Content
- 6. © Tubi, proprietary and confidential 6 Core Idea: Recommend items based upon the previous interactions of the users. Popular approaches: Collaborative Filtering. Recommender system Buy People Also Buy Recommend
- 7. © Tubi, proprietary and confidential Exploration
- 8. © Tubi, proprietary and confidential 8 Production bias: Only get feedback from recommended items. Only show titles with sufficient feedback. Why do we need Exploration ? Problem: ● How to cold start new titles/users ? ● How to handle changing user tastes ? ● How to handle changing title popularities ? Recommend ML Model
- 9. © Tubi, proprietary and confidential 9 Exploration-Exploitation Tradeoff: Explore unknown items to gather feedback or exploit current best item Exploration in Recommendation
- 10. © Tubi, proprietary and confidential 10 P(Epsilon): Explore P(1- Epsilon): Exploit Epsilon-Greedy
- 11. © Tubi, proprietary and confidential 11 Optimism in the Face of Uncertainty Principle: “Choose your actions as if the environment is as nice as plausible” Optimism leads to exploration - reward from the unknown arm possibly greater Upper Confidence Bound Arm 1 Arm 2 X X X X X X X Mu 1 Mu 2 UCB1 UCB2 Optimistic estimate = Mu + CI
- 12. © Tubi, proprietary and confidential 12 Maintain a posterior distribution over the parameters for each action. Sample the estimated reward Recommend the item with the highest estimated reward Update the posterior. Thompson Sampling Arm1 Arm2 Arm3 Arm4
- 13. © Tubi, proprietary and confidential 13 No assumptions on a reward distribution. Reward non-stationary. Based upon Exponential Weighted algorithm “Hedge” Algorithms slower in practice. Adversarial setting Exp3
- 14. © Tubi, proprietary and confidential 14 Additional context provided to the learner No single arm is good in all cases Classical MAB: Compete against the best arm Contextual MAB: Compete against the best “policy” Contextual Bandits Context Past Viewing History Time of Day Country/DMA Recommend
- 15. © Tubi, proprietary and confidential 15 Many practical challenges ● Arms keep changing ● Non-stationary rewards ● Budget constraints ● Slate of recommendations ● Feedback loops Bandits in the real world
- 16. © Tubi, proprietary and confidential Diversity
- 17. © Tubi, proprietary and confidential 17 Correlations: Ordering items by relevance score could result in correlations between nearby items Why do we need Diversity? Problems: ● Utility of the “set” of recommendations not maximized. ● Real estate better utilized by other title
- 18. © Tubi, proprietary and confidential 18 “Remove redundant items to replace with alternatives that the user will enjoy concurrently.” Compute a diversity score Methods based on coverage and redundancy Diversity in Recommendations rerankScore = (1 - lambda) * relevancyScore + lambda * diversityScore
- 19. © Tubi, proprietary and confidential 19 YouTube homepage diversification DPP - characterized by the determinant of some function. Let Relevancy score = P1, P2, … PN; Similarity score = Sij Determinantal Point Process P1 S12 S13 S14 S12 P2 S23 S24 S13 S23 P3 S34 S14 S24 S34 P4 Determinant
- 20. © Tubi, proprietary and confidential 20 Exploration Primary Goal: To understand user preferences for items as there is imperfect information. Exploration is still needed for uncorrelated recommendations. The story of Exploration and Diversity Diversity Primary Goal: Get rid of correlated recommendations for better user experience. Diversity is still needed when we know the user preferences accurately. Exploration can be designed to achieve diversification and diversification can help achieve exploration!
- 21. © Tubi, proprietary and confidential 21 Lattimore, T., & Szepesvári, C. (2020). Bandit Algorithms. Cambridge: Cambridge University Press. doi:10.1017/9781108571401 Wilhelm, Mark & Ramanathan, Ajith & Bonomo, Alexander & Jain, Sagar & Chi, Ed & Gillenwater, Jennifer. (2018). Practical Diversified Recommendations on YouTube with Determinantal Point Processes. 2165-2173. 10.1145/3269206.3272018. References
- 22. © Tubi, proprietary and confidential 22 Thank You! We are hiring! Email: jkawale@tubi.tv Twitter: jayakawale