The Effect of Different Set-based Visualizations on User Exploration of Recommendations

1. The Effect of Different Set-based Visualizations on User Exploration of Recommendations KatrienVerbert, KU Leuven Denis Parra, PUC Chile Peter Brusilovsky, University of Pittsburgh IntRSWorkshop at RecSys2014, Foster City, CA, USA

2. Outline •Context of this Work in RecSysresearch •Set-based Visual Interfaces for User Exploration –TalkExplorer: Multimode graph –SetFusion: Venn diagram •Meta-Analysis •Summary & Future Work 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 2

3. INTRODUCTION Recommender Systems: Introduction & Motivation 3 * Danboard(Danbo): Amazon’s cardboard robot, in these slides represents a recommender system *

4. Recommender Systems (RecSys) Systems that help people (or groups) to find relevant items in a crowded item or information space (McNeeet al. 2006) 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 4

5. Challenges of RecSysAddressed Here Traditionally, RecSys has focused on producing accurate recommendation algorithms. In this research, we address these challenges: 1.HCI: Implementation of visualizations that enhance users’ exploration of the items suggested. 2.Recommendation Tasks: Tackling exploration of recommendations, not only rating prediction or Top-N. 3.Meta-Analysis: Comparing results of different studies to generalizeresults. 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 5

6. Research Platform •The studies were conducted using Conference Navigator, a Conference Support System •Our goal was recommending conference talks 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 6 Program Proceedings Author List Recommendations http://halley.exp.sis.pitt.edu/cn3/

7. RELATED WORK OF VISUAL INTERFACES FOR RECSYS Previous research related to this work / Motivating results from TalkExplorer study 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 7

8. PeerChooser–CF movies 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 8 O'Donovan, J., Smyth, B., Gretarsson, B., Bostandjiev, S., & Höllerer, T. (2008, April). PeerChooser: visual interactiverecommendation

9. SmallWorlds–CF Social 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 9 Gretarsson, B., O'Donovan, J., Bostandjiev, S., Hall, C., & Höllerer, T. (2010, June). Smallworlds: Visualizingsocial recommendations.

10. TasteWeights–Hybrid Recommender 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 10 Bostandjiev, S., O'Donovan, J., & Höllerer, T. (2012, September). Tasteweights: a visual interactivehybridrecommendersystem

11. TALKEXPLORER: A GRAPH-BASED INTERACTIVE RECOMMENDER 11

12. TalkExplorer–IUI 2013 •Adaptation of Aduna Visualization to CN •Main research question: Does fusion(intersection) of contexts of relevance improve user experience? 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 12

13. TalkExplorer-I 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 13 Entities Tags, Recommender Agents, Users

14. TalkExplorer-II 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 14 Recommender Recommender Cluster with intersection of entities Cluster (of talks) associated to only one entity •Canvas Area: Intersections of Different Entities User

15. TalkExplorer-III 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 15 Items Talks explored by the user

16. Our Assumptions •Itemswhicharerelevant in more that one aspect could be more valuable to the users •Displaying multiple aspects of relevance visually is important for the users in the process of item’s exploration 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 16

17. TalkExplorerStudies I & II •Study I –Controlled Experiment: Users were asked to discover relevant talks by exploring the three types of entities: tags, recommender agents and users. –Conducted at Hypertext and UMAP 2012 (21 users) –Subjects familiar with Visualizations and Recsys •Study II –Field Study: Users were left free to explore the interface. –Conducted at LAK 2012 and ECTEL 2013 (18 users) –Subjects familiar with visualizations, but not much with RecSys 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 17

18. Evaluation: Intersections & Effectiveness •What do we call an “Intersection”? •We used #explorations on intersections and their effectiveness, defined as: Effectiveness = |푏표표푘푚푎푟푘푒푑푖푡푒푚푠| |푖푛푡푒푟푒푠푒푐푡푖표푛푠푒푥푝푙표푟푒푑| 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 18

19. Results of Studies I & II •Effectiveness increases with intersections of more entities •Effectiveness wasn’t affected in the field study (study 2) •… but exploration distribution was affected 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 19

20. Drawback: Visualizing Intersections Clustermap Venn diagram •Venn diagram: more natural way to visualize intersections 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 20

21. SETFUSION: VENN DIAGRAM FOR USER-CONTROLLABLE INTERFACE 21

22. SetFusion–IUI 2014 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 22

23. SetFusionI 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 23 Traditional Ranked List Papers sorted by Relevance. It combines 3 recommendation approaches.

24. SetFusion-II 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 24 Sliders Allow the user to control the importance of each data source or recommendation method Interactive Venn Diagram Allows the user to inspect and to filter papers recommended. Actions available: -Filter item list by clicking on an area -Highlight a paper by mouse-over on a circle -Scroll to paper by clicking on a circle -Indicate bookmarked papers

25. SetFusion – UMAP 2012 • Field Study: let users freely explore the interface 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 25 - ~50% (50 users) tried the SetFusion recommender - 28% (14 users) bookmarked at least one paper - Users explored in average 14.9 talks and bookmarked 7.36 talks in average. A AB ABC AC B BC C 15 7 9 26 18 4 17 16% 7% 9% 27% 19% 4% 18% Distribution of bookmarks per method or combination of methods

26. META-ANALYSIS Description and Analysis of the results of the 3 user studies

27. TalkExplorervs. SetFusion •Comparing distributions of explorations 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 27 In studies 1 and 2 over talkEplorerwe observed an important change in the distribution of explorations.

28. TalkExplorervs. SetFusion •Comparing distributions of explorations 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 28 Comparing the field studies: -In TalkExplorer, 84% of the explorations over intersections were performed over clusters of 1 item -In SetFusion, was only 52%, compared to 48% (18% + 30%) of multiple intersections, diff. not statistically significant

29. CONCLUSIONS & FUTURE WORK

30. Summary of this Talk •We presented two implementations of visual interactive interfaces that tackle exploration on a recommendation setting •We showed that intersections of several contexts of relevance help to discover relevant items •The visual paradigm used can have a strong effect on user behavior: we need to keep working on visual representation that promote exploration without increasing the cognitive load over the users 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 30

31. Limitations & Future Work •Apply our approach to other domains (fusion of data sources or recommendation algorithms) •For SetFusion, find alternatives to scale the approach to more than 3 sets, potential alternatives: –Clustering and –Radial sets •Consider other factors that interact with the user satisfaction: –Controllability by itself vs. minimum level of accuracy 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 31

32. THANKS! QUESTIONS? DPARRA@ING.PUC.CL

33. Mixed Hybridization: Item Score 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 33 M: is the set of all methods available to fuse rankreci,mj: rank–position in the list of a recommended item reci: recommended method i mj, : recommendation method j Wmj:weight given by the user to the method mjusing the controllable interface |Mreci| represents the number of methods by which item reciwas recommended Slider weight

34. Hybridization Methods (Burke 2002) 10/06/2014 Verbet, Parra, & Brusilovsky .~ IntRs as RecSys 2014 34 Hybridization Description Weighted The scores (or votes) of several recommendation techniques are combined together to produce a single recommendation. Switching The system switches between recommendation techniques depending on the current situation. Mixed Recommendations from several different recommenders are presented at the same time Feature combination Features from different recommendation data sources are thrown together into a single recommendation algorithm Cascade One recommender refines the recommendations given by another. Feature augmentation Output from one technique is used as an input feature to another. Meta-level The model learned by one recommender is used as input to another.