Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Social Aspects of Interactive Recommender Systems

Social Aspects of Interactive Recommender Systems, invited talk at the SoAPS workshop, ECIR conference 2018, Grenoble, France.

  • Identifiez-vous pour voir les commentaires

Social Aspects of Interactive Recommender Systems

  1. 1. Social aspects of interactive RecSys: Bridging the gap between predictive algorithms and interactive user interfaces Denis Parra, Assistant Professor CS Department School of Engineering Pontificia Universidad Católica deChile SoAPS Workshop at ECIR 2018, March 26th 2018 Funded by :
  2. 2. Personal Introduction • 2008-2013: PhD at U. of Pittsburgh • 2013 – now: CS Department, PUC (Santiago, Chile) March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 2 8,280 Km
  3. 3. Outline • Quick intro to – RecommenderSystems – Visualization in RecommenderSystems • Survey of our work on Visualization and Interaction on RecSys – With highlights to social aspects of the results • Summary & Discussion March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 3
  4. 4. In Collaboration with • Peter Brusilovsky (University of Pittsburgh, USA) • Katrien Verbert (KU Leuven, Belgium) • Christoph Trattner (University of Bergen, Norway) • Chaoli Wang (Notre Dame University, USA) • Ivania Donoso (alumni, PUC Chile) • María Sepúlveda (MSc student, PUC Chile) March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 4
  5. 5. INTRODUCTIONTORECSYS Recommender Systems * Danboard (Danbo):Amazon’s cardboard robot, in these slides it represents a recommender system *
  6. 6. Recommender Systems (RecSys) Systems that help (groups of) people to find relevant items in a crowdeditem or information space(MacNee et al. 2006) March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 6
  7. 7. Why do we care about RecSys? • Nowadays, several domains& applications require people to make decisions among a large set of items. March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 7
  8. 8. A lil’ bit of History • First recommender systems were built at the beginning of 90’s (Tapestry, GroupLens, Ringo) • Online contests, such as the Netflix prize, grew the attention on recommender systems beyond Computer Science (2006-2009) March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 8
  9. 9. The Recommendation Problem • A popular way of presenting the recommendation problem was rating prediction (Netflix prize) • How good is my prediction? Item 1 Item 2 … Item m User 1 1 5 4 User 2 5 1 ? … User n 2 5 ? Predict! March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 9
  10. 10. Traditional Recommendation Methods • Without covering all possible methods, the two most typical classifications on recommender algorithms are Classification 1 Classification 2 - Collaborative Filtering - Content-based Filtering - Hybrid - Memory-based - Model-based March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 10
  11. 11. Collaborative Filtering (User-based KNN) • Find like-minded people to recommend 5 4 4 2 1 5 4 4 Active user User_1 User_2 ∑ ∑ ⊂ ⊂ −⋅ += )( )( ),( )(),( ),( uneighborsn uneighborsn nni u nuuserSim rrnuuserSim riupred2 3 4 2 Item 1 Item 2 Item 3 March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 11
  12. 12. Content-Based Filtering • Can be traced back to techniques from IR, where the User Profile represents a query. user_profile = {w_1, w_2, …., w_3} using TF-IDF, weighting Doc_1 = {w_1, w_2, …., w_3} Doc_2 = {w_1, w_2, …., w_3} Doc_3 = {w_1, w_2, …., w_3} Doc_n = {w_1, w_2, …., w_3} 5 4 5 March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 12
  13. 13. Hybridization • Combine previous methods to overcome their weaknesses (Burke, 2002) March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 13
  14. 14. Model-based: Matrix Factorization Latent vector of the item Latent vector of the user SVD ~ Singular Value Decomposition March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 14
  15. 15. Other paradigms and techniques • Recommendation as a graph problem: – PersonalizedPageRank (Kamvar et al, 2010), (Santos et al 2016), etc. • Recommendation as a ranking problem: – Karatzoglou et al. (2013), Shi et al. (2014), Macedo et al. (2015), etc. • Deep learning methods: – MetaProd2Vecby Vasile et al. (2016), YouTube Recommendationsby Covington et al. (2016), etc. March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 15
  16. 16. Are accuracy metrics the goal of RecSys? • Some works started pointing out that small improvements in RMSE did not have a proportional improvement in user satisfaction March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 16
  17. 17. MovieLens: Traditional RecSys Interface March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 17
  18. 18. PeerChooser (2008) Controllability in CF March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 18 O’Donovan et al. “PeerChooser: Visual Interactive Recommendation” (2008)
  19. 19. SmallWorlds: Expanded Explainability March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 19 Gretarsson et al. “SmallWorlds:Visualizing social recommendations” (2010)
  20. 20. TasteWeights: Hybrid Control and Inspect Bostandjev et al. “TasteWeights:A Visual Interactive Hybrid Recommender System” (2012) Controllability: Sliders that let users control the importance of preferences and contexts Inspectability: lines that connect recommended items with contexts and user preferences March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 20
  21. 21. IUI 2017 • Loepp et al. (2017) March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 21
  22. 22. More Details? Check our survey March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 22 He, C., Parra, D., & Verbert, K. (2016). Interactive recommender systems: a survey of the state of the art and future research challenges andopportunities.Expert Systems with Applications, 56, 9-27.
  23. 23. My Take on RecSys Research (2009 ~) March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 23
  24. 24. IUI 2013 IUI 2014 UMAP 2016 E.I.-VDA 2018 IUI 2018 Visual & interactive RecSys interfaces March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 24 CNVis EpistAid TalkExplorer SetFusion Moodplay
  25. 25. March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 25
  26. 26. TalkExplorer – IUI 2013 • Adaptation of Aduna Visualization to CN • Main researchquestion: Does fusion (intersection) of contextsof relevance improve user experience with RecSys? March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 26
  27. 27. Research Platform • The studies were conducted using Conference Navigator, a Conference Support System • Our goal was recommending conference talks Program Proceedings Author List Recommendations http://halley.exp.sis.pitt.edu/cn3/ March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 27
  28. 28. TalkExplorer - I Entities Tags, Recommender Agents, Users March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 28
  29. 29. TalkExplorer - II Recommender Recommender Cluster with intersecti on of entities Cluster (of talks) associated to only one entity • Canvas Area: Intersections of Different Entities User March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 29
  30. 30. TalkExplorer - III Items Talks explored by the user March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 30
  31. 31. Evaluation: Intersections & Effectiveness • What do we call an “Intersection”? • We used #explorations on intersections and their effectiveness, defined as: Effectiveness = March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 31
  32. 32. Our Assumptions • Items which are relevant in more that one aspect could be more valuable to the users • Displaying multiple aspects of relevance visually is important for the users in the process of item’s exploration March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 32
  33. 33. TalkExplorer Studies I & II • Study I – ControlledExperiment:Userswere asked todiscover relevant talksby exploringthe three types of entities:tags, recommenderagents and users. – Conductedat HypertextandUMAP 2012(21 users) – Subjectsfamiliarwith Visualizationsand Recsys • Study II – FieldStudy: Users were left free to explore the interface. – Conductedat LAK 2012 and ECTEL 2013 (18 users) – Subjectsfamiliarwith visualizations, but not much with RecSys March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 33
  34. 34. Results of Studies I & II • Effectiveness increases with intersections of more entities • Effectiveness wasn’t affected in the field study (study 2) • … but exploration distribution was affected March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 34
  35. 35. More detail on entities March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 35
  36. 36. More detail on entities March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 36 • Despite not being the most effective, the user (social) entity attracted by far more explorations from users.
  37. 37. Social ~ Trust in RecSys • O'Donovan, John, and Barry Smyth. "Trust in recommender systems," IUI, 2005. • Golbeck, Jennifer, and James Hendler. "Filmtrust: Movie recommendations using trust in web-based social networks," ComNet, 2006. • Jamali, Mohsen, and Martin Ester. "A matrix factorization technique with trust propagation for recommendation in social networks." RecSys, 2010. March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 37
  38. 38. SetFusion – IUI 2014 March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 38
  39. 39. SetFusion I Traditional Ranked List Papers sorted by Relevance. It combines 3 recommendation approaches. March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 39
  40. 40. SetFusion - II Sliders Allow the user to control the importance of each data source or recommendation method Interactive Venn Diagram Allows the user to inspect and to filter papers recommended. Actions available: - Filter item list by clicking on an area - Highlight a paper by mouse-over on a circle - Scroll to paper by clicking on a circle - Indicate bookmarked papers March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 40
  41. 41. Study : iConference March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 41 A B C • A and C: Social • B: Content-based
  42. 42. Rating per method – Effect of Visuals March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 42
  43. 43. Rating per method – Effect of Visuals March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 43 No significant difference
  44. 44. Rating per method – Effect of Visuals March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 44 Hybrid algorithm + visualization yield the only significant difference
  45. 45. Summary & Conclusions • The combination of several sources of relevance has an impact on recommendation, being the social aspect among the most relevant. • The visual paradigm combined with social aspects used can have a significant effect on user behavior. • We need to keep working on visual representations that promote exploration without decreasing recommendation accuracy. March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 45
  46. 46. MOODFROM SOCIAL SOURCES FOR MUSICRECOMMENDATION
  47. 47. Moodplay • MoodPlay – With Ivana Andjelkovic& John O’Donovan (UCSB) March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 47 Andjelkovic,I., Parra, D., & O'Donovan,J. (2016). Moodplay:Interactive Mood-based Music Discovery and Recommendation.In Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization (pp. 275-279).ACM.
  48. 48. MoodPlay March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 48
  49. 49. System Architecture March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 49
  50. 50. Emotion Models • Modelo de emociones de Russel (1980) • GEMS (2008) March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 50
  51. 51. Moods and Music: the GEMS model March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 51
  52. 52. Components • Colors: GEMS emotions • Dots: artists • Zooming and Panning • Playing & Recommendations March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 52
  53. 53. Components • Colors: GEMS emotions • Dots: artists • Zooming and Panning • Playing & Recommendations March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 53
  54. 54. Components • Colors: GEMS emotions • Dots: artists • Zooming and Panning • Playing & Recommendations March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 54
  55. 55. Components • Colors: GEMS emotions • Dots: artists • Zooming and Panning • Playing & Recommendations March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 55
  56. 56. Emotion-Aware Recommendation • Using several Web APIs, we collected users’ perception of mood associated to artists. • Then, using artists as input, we calculate recommendations of similarly perceived artists March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 56
  57. 57. MoodPlay March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 57 • Our results indicate that users’ reported mood and artist mood have an effect on people satisfaction with the system. • Try it at http://moodplay.pythonanywhere.com
  58. 58. SOME CHALLENGES & CONCLUSION
  59. 59. March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 59 Interactive Relevance Feedback Interface for evidence–based Medicine Ivania Donoso-Guzmán Denis Parra Honorable mention for best paper award ACM IUI 2018
  60. 60. Belleret al (2013). … takes from 6 to 12 meses Process for answering a clinical question 60 March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018)
  61. 61. 61 March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) EpistAid
  62. 62. Model Interfaz N Documentos Vistos Ground Truth Recall Precision F-1 score Recall BM25 Non-Viz 12 0.66+/-0.08 0.52 +/- 0.06 0.58 +/- 0.07 0.20 +/- 0.04 BM25 Viz 11 0.71+/-0.06 0.64 +/- 0.04 0.64 +/- 0.03 0.18 +/- 0.02 Rocchio Non-Viz 11 0.65+/- 0.08 0.73 +/- 0.02 0.65 +/- 0.05 0.21 +/- 0.04 Rocchio Viz 12 0.77 +/- 0.06 0.67 +/- 0.01 0.70 +/- 0.03 0.23 +/- 0.03 Promising Results 62 March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018)
  63. 63. Future Work March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 63 - CollaborativeModel: - Whichis the best way to providea collaborative human-in-the-loopinterfacefor Evidence-Based Medicine? - … Ideas from research on collaborative web search, for instance: Yue et al. “Influences on query reformulation in collaborative web search”. IEEE Computer Magazine, 2014.
  64. 64. IEEE VIS 2017 – Panel ML & Vis ? March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 64
  65. 65. Deep Learning and Social on RecSys • Lei, Chenyi, et al. "Comparative deep learning of hybrid representations for image recommendations." CVPR. 2016. March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 65
  66. 66. Deep Learning and Social on RecSys • Lei, Chenyi, et al. "Comparative deep learning of hybrid representations for image recommendations." CVPR. 2016. March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 66 - How to make better sense of the social embeddings for explaining recommendations?
  67. 67. IEEE VIS 2017 – Panel ML & Vis ? March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 67
  68. 68. Conclusion March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 68 • The social aspects are indeed relevant in recommendation systems, and their effect on prediction accuracyhas been already studied. • The studies presented in this talk show that combining interactive visualizations with social and other relevance signal can have an important effect on users’ perception of recommendations. • I invite researcher to further study the connections between social aspects, visualization and their effect on recommender systems.
  69. 69. THANKS! dparra@ing.puc.cl

×