Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
The Effect of Explanations and Algorithmic Accuracy
on
Visual Recommender Systems of Artistic Images
Pontificia Universida...
The Effect of Explanations and Algorithmic Accuracy
on
Visual Recommender Systems of Artistic Images
(candidate to best pa...
The Online Artwork Market
• Online artwork market: Growing since 2008,
despite global crises!
• In 2011, art received $11....
Related Work: Art Recommendation
March 17th 2019 Dominguez et al ~ ACM IUI 2019 4
• Previous art recommendationprojects da...
Related Work: Art Recommendation
March 17th 2019 Dominguez et al ~ ACM IUI 2019 5
• Previous art recommendationprojects da...
Related Work: Art Recommendation
• Previous art recommendationprojects date for as long as 2007, such
as the CHIP project ...
Our Approach to Image Recommendation
• Since 2017 we have been working on recommending art
images, using mostly content-ba...
Open Questions from Content-Based RecSys
• We learned that visual features from DNNs perform better than
attractiveness vi...
Open Questions from Content-Based RecSys
• We learned that visual features from DNNs perform better than
attractiveness vi...
Explainability in Recommender Systems
• Explaining recommendations is a well-established and
active area of research, but ...
Gaining Inspiration from DARPA XAI
March 17th 2019 Dominguez et al ~ ACM IUI 2019 11
DARPA XAI (Gunning, 2017)
Decision Tr...
DARPA XAI inspiration
March 17th 2019 Dominguez et al ~ ACM IUI 2019 12
DARPA XAI (Gunning, 2017)
Decision Tree
DNN
Our re...
Our General Research Question
• Explainable &
Transparent UIs are well
perceived by users in
previous RecSys
research, but...
Research Questions
• RQ1. Given three different types of interfaces, one baseline interface without
explanations and two w...
Research Questions
• RQ1. Given three different types of interfaces, one baseline interface without
explanations and two w...
Research Questions
• RQ1. Given three different types of interfaces, one baseline interface without
explanations and two w...
Materials and Methodology
1. We collect image data from UGallery, an online store of
physical artworks (paintings and phot...
Data: UGallery
• Online Artwork
Store, based on CA,
USA.
• Mostly sales one-
of-a-kind physical
artwork.
March 17th 2019 D...
CB RecSys algorithm: Visual Features
• (DNN) Deep Neural
Networks
• (AVF) Attractiveness-
based
March 17th 2019 Dominguez ...
Visual Features: DNN vs. AVF
• Deep Neural Networks (DNN):
we used an AlexNet DNN
(pre-trained with ImageNet
ILSVRC 2012 d...
Visual Features: AVF
• Average brightness,
• Saturation,
• Sharpness,
• Entropy,
• RGB-contrast,
• Colorfulness,
• Natural...
UGallery: Making Recommendations
• Scoring items based on cosine similarity between user
model and item model:
Generating Explanations
March 17th 2019 Dominguez et al ~ ACM IUI 2019 23
Friedrich, G., & Zanker, M. (2011). A
taxonomy f...
White-box Explanation (AVF)
March 17th 2019 Dominguez et al ~ ACM IUI 2019 24
Black-box explanation (DNN and AVF)
March 17th 2019 Dominguez et al ~ ACM IUI 2019 25
Study Procedure
March 17th 2019 Dominguez et al ~ ACM IUI 2019 26
Study Procedure
March 17th 2019 Dominguez et al ~ ACM IUI 2019 27
Study Procedure
March 17th 2019 Dominguez et al ~ ACM IUI 2019 28
Preference Elicitation
• We collect user preferences from a Pinterest-like interface
March 17th 2019 Dominguez et al ~ ACM...
Study Procedure
March 17th 2019 Dominguez et al ~ ACM IUI 2019 30
Interface 1: no explanation, no transparency
March 17th 2019 Dominguez et al ~ ACM IUI 2019 31
Study Procedure
March 17th 2019 Dominguez et al ~ ACM IUI 2019 32
Interface 2: explainable, no transparency
March 17th 2019 Dominguez et al ~ ACM IUI 2019 33
Study Procedure
March 17th 2019 Dominguez et al ~ ACM IUI 2019 34
Interface 3: explainable & transparent
March 17th 2019 Dominguez et al ~ ACM IUI 2019 35
Study Procedure
March 17th 2019 Dominguez et al ~ ACM IUI 2019 36
Evaluation & Results
• Post-algorithm survey aspects (agreement1-100)
– Explainable:I understood why the art images were r...
Demo
March 17th 2019 Dominguez et al ~ ACM IUI 2019 38
Evaluation & Results
Study on Amazon Mechanical Turk:
● 121 valid users completed correctly the study.
● Task took them ar...
Results
March 17th 2019 Dominguez et al ~ ACM IUI 2019 40
Interface 1: UI without explanation
Interface 2: UI with example...
Results
March 17th 2019 Dominguez et al ~ ACM IUI 2019 41
7 dimensions evaluated, for DNN and AVF (scale 1-100):
Perceptio...
Results
• Result 1: DNN better or equal than AVF except on diversity
March 17th 2019 Dominguez et al ~ ACM IUI 2019 42
- A...
Results
• Result 2: Explainable interfaces increase perception of
explainability.
March 17th 2019 Dominguez et al ~ ACM IU...
Results
• Result 2: Explainable interfaces increase perception of
explainability.
March 17th 2019 Dominguez et al ~ ACM IU...
Evaluation & Results
• Result 2: Perception of relevance changes just by adding
explanations -> User Interface really matt...
Evaluation & Results
• Result 3: No difference in Trust between DNN and AVF in I1
(without explanations)
March 17th 2019 D...
Evaluation & Results
• Result 4: Without explanations, No difference between DNN
and AVF upon intention of use and satisfa...
General Model: Knijnenburg Framework
March 17th 2019 Dominguez et al ~ ACM IUI 2019 48
Knijnenburg,B. P., & Willemsen,M. C...
Visual Art RecSys Explanation Model
March 17th 2019 Dominguez et al ~ ACM IUI 2019 49
Visual Art RecSys Explanation Model
March 17th 2019 Dominguez et al ~ ACM IUI 2019 50
- Compared to the baseline interface...
Visual Art RecSys Explanation Model
March 17th 2019 Dominguez et al ~ ACM IUI 2019 51
- Compared to the
baseline AVF, DNN
...
Visual Art RecSys Explanation Model
March 17th 2019 Dominguez et al ~ ACM IUI 2019 52
- Perceived Effort (Insecurity, rush...
Visual Art RecSys Explanation Model
March 17th 2019 Dominguez et al ~ ACM IUI 2019 53
- Satisfaction is strongly
affected ...
Conclusion
• DNN features performed better than AVF features, probably because
AlexNet is able to capture more complex pat...
Future Work
• (limitation) If additional transparency is provided in an
explanation, make sure the explanation is actually...
Acknowledgment
• The were funded by PUC Chile, Conicyt, Fondecyt grant
11150783, as well as by the Millennium Institute fo...
THANKS!
Denis Parra
Assistant Professor
Pontificia Universidad Católicade Chile
dparras@uc.cl
References
● Vicente Dominguez, Pablo Messina, Denis Parra, Domingo Mery, Christoph Trattner, and
Alvaro Soto. 2017. Compa...
Prochain SlideShare
Chargement dans…5
×

The Effect of Explanations & Algorithmic Accuracy on Visual Recommender Systems of Artistic Images - ACM IUI 2019

My presentation at ACM Conference on Intelligent User Interfaces (IUI 2019) "The Effect of Explanations & Algorithmic Accuracy on Visual Recommender Systems of Artistic Images"

  • Identifiez-vous pour voir les commentaires

  • Soyez le premier à aimer ceci

The Effect of Explanations & Algorithmic Accuracy on Visual Recommender Systems of Artistic Images - ACM IUI 2019

  1. 1. The Effect of Explanations and Algorithmic Accuracy on Visual Recommender Systems of Artistic Images Pontificia UniversidadCatólica de Chile (PUC Chile) Vicente Domínguez Pablo Messina Ivania Donoso-Guzmán Denis Parra
  2. 2. The Effect of Explanations and Algorithmic Accuracy on Visual Recommender Systems of Artistic Images (candidate to best paper) Pontificia UniversidadCatólica de Chile (PUC Chile) Vicente Domínguez Pablo Messina Ivania Donoso-Guzmán Denis Parra
  3. 3. The Online Artwork Market • Online artwork market: Growing since 2008, despite global crises! • In 2011, art received $11.57 billion in total global annual revenue, over $2 billion versus 2010 (*forbes) • Recommender systems could play an important role, just like in other e-commerce industries. [forbes] https://www.forbes.com/sites/abigailesman/2012/02/29/ the- worlds- strongest- economy- the- global- art- market/ (2012) March 17th 2019 Dominguez et al ~ ACM IUI 2019 3
  4. 4. Related Work: Art Recommendation March 17th 2019 Dominguez et al ~ ACM IUI 2019 4 • Previous art recommendationprojects date for as long as 2007, such as the CHIP project to recommend paintings from Rijksmuseum. (Aroyo et al., 2007) • Some approaches have used: – Ratings (Aroyo et al., 2007), – Tags, Keywords or other textual features (Semeraro et al. 2012) – Traditional visual features (van den Broek et al., 2006) • Deep Learning (DL) have revolutionized visual feature learning, but few works use them for recommending art (Het et al. 2016,Messina et al. 2018)
  5. 5. Related Work: Art Recommendation March 17th 2019 Dominguez et al ~ ACM IUI 2019 5 • Previous art recommendationprojects date for as long as 2007, such as the CHIP project to recommend paintings from Rijksmuseum. (Aroyo et al., 2007) • Some approaches have used: – Ratings (Aroyo et al., 2007), – Tags, Keywords or other textual features (Semeraro et al. 2012) – Traditional visual features (van den Broek et al., 2006) • Deep Learning (DL) have revolutionized visual feature learning, but few works use them for recommending art (Het et al. 2016,Messina et al. 2018)
  6. 6. Related Work: Art Recommendation • Previous art recommendationprojects date for as long as 2007, such as the CHIP project to recommend paintings from Rijksmuseum. (Aroyo et al., 2007) • Some approaches have used: – Ratings (Aroyo et al., 2007), – Tags, Keywords or other textual features (Semeraro et al. 2012) – Traditional visual features (van den Broek et al., 2006) • Deep Learning (DL) have revolutionized computer vision, but few works use DL methods for recommending art (He et al. 2016,Messina et al. 2018) March 17th 2019 Dominguez et al ~ ACM IUI 2019 6
  7. 7. Our Approach to Image Recommendation • Since 2017 we have been working on recommending art images, using mostly content-based methods. • Four papers published: – ACM DLRS 2017: Dominguez,V.,Messina,P., Parra, D., Mery, D., Trattner,C., & Soto,A. (2017,August).Comparing Neural and Attractivenessbased Visual Features forArtwork Recommendation.In Proceedingsof the2nd Workshop on Deep Learning for RecommenderSystems(pp.55-59).ACM. – UMUAI 2018: Messina,P., Dominguez,V.,Parra, D., Trattner,C., & Soto, A. (2018).Content- based artwork recommendation:integrating paintingmetadatawith neural and manually- engineered visual features.UserModeling and User-Adapted Interaction,1-40. – Two additional workshop papers in ACM RecSys 2018. March 17th 2019 Dominguez et al ~ ACM IUI 2019 7
  8. 8. Open Questions from Content-Based RecSys • We learned that visual features from DNNs perform better than attractiveness visual features. March 17th 2019 Dominguez et al ~ ACM IUI 2019 8 Average brightness Saturation Sharpness Entropy RGB-contrast Colorfulness Naturalness Predictive Accuracy Attractiveness visual features Deep Learning visual features
  9. 9. Open Questions from Content-Based RecSys • We learned that visual features from DNNs perform better than attractiveness visual features, but they are harder to explain. March 17th 2019 Dominguez et al ~ ACM IUI 2019 9 Predictive Accuracy Explainability Average brightness Saturation Sharpness Entropy RGB-contrast Colorfulness Naturalness Attractiveness visual features Deep Learning visual features
  10. 10. Explainability in Recommender Systems • Explaining recommendations is a well-established and active area of research, but there is little research on explaining image recommendations. March 17th 2019 Dominguez et al ~ ACM IUI 2019 Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000). Explaining collaborative filtering recommendations. In Proceedings of the 2000 ACM conference on Computer supported cooperative work (pp. 241-250) 10 Tintarev, N., & Masthoff, J. (2015). Explaining recommendations: Design and evaluation. In Recommender systems handbook (pp. 353- 382). Springer, Boston, MA.
  11. 11. Gaining Inspiration from DARPA XAI March 17th 2019 Dominguez et al ~ ACM IUI 2019 11 DARPA XAI (Gunning, 2017) Decision Tree DNN
  12. 12. DARPA XAI inspiration March 17th 2019 Dominguez et al ~ ACM IUI 2019 12 DARPA XAI (Gunning, 2017) Decision Tree DNN Our research AVF DNN VF Explainability Potential PredictiveAccuracy Is there an effect on user perception if we provide a UI with explanations?
  13. 13. Our General Research Question • Explainable & Transparent UIs are well perceived by users in previous RecSys research, but: Does it matter to UX if you do not produce accurate enough predictions? March 17th 2019 Dominguez et al ~ ACM IUI 2019 13 AVF DNN VF Explainability Potential PredictiveAccuracy - UI w/explanation based on examples - UI w/explanation based on examples or visual features
  14. 14. Research Questions • RQ1. Given three different types of interfaces, one baseline interface without explanations and two with explanations but different levels of transparency, which one is perceived as most useful? • RQ2. Furthermore, based on the visual content-based recommenderalgorithm chosen (DNN or AVF), are there observable differences in how the three interfaces are perceived? • RQ3. How do independent variables such as algorithm, explainable interface and domain knowledge interact in order to explain the user experience with the recommender system in terms of perception of relevance, diversity, explainabilityand user satisfaction? March 17th 2019 Dominguez et al ~ ACM IUI 2019 14
  15. 15. Research Questions • RQ1. Given three different types of interfaces, one baseline interface without explanations and two with explanations but different levels of transparency, which one is perceived as most useful? • RQ2. Furthermore, based on the visual content-based recommenderalgorithm chosen (DNN or AVF), are there observable differences in how the three interfaces are perceived? • RQ3. How do independent variables such as algorithm, explainable interface and domain knowledge interact in order to explain the user experience with the recommender system in terms of perception of relevance, diversity, explainabilityand user satisfaction? March 17th 2019 Dominguez et al ~ ACM IUI 2019 15
  16. 16. Research Questions • RQ1. Given three different types of interfaces, one baseline interface without explanations and two with explanations but different levels of transparency, which one is perceived as most useful? • RQ2. Furthermore, based on the visual content-based recommenderalgorithm chosen (DNN or AVF), are there observable differences in how the three interfaces are perceived? • RQ3. How do independent variables such as algorithm, explainable interface and domain knowledge interact in order to explain the user experience with the recommender system in terms of perception of relevance, diversity, explainabilityand user satisfaction? March 17th 2019 Dominguez et al ~ ACM IUI 2019 16
  17. 17. Materials and Methodology 1. We collect image data from UGallery, an online store of physical artworks (paintings and photographs). 2. We implement a system which records user preferences and generates recommendations. Some of these recommendations will have explanations. 3. We conduct a user study in Amazon Mechanical Turk to assess the impact of algorithmic accuracy and explainable UI on several user dimensions. March 17th 2019 Dominguez et al ~ ACM IUI 2019 17
  18. 18. Data: UGallery • Online Artwork Store, based on CA, USA. • Mostly sales one- of-a-kind physical artwork. March 17th 2019 Dominguez et al ~ ACM IUI 2019 18
  19. 19. CB RecSys algorithm: Visual Features • (DNN) Deep Neural Networks • (AVF) Attractiveness- based March 17th 2019 Dominguez et al ~ ACM IUI 2019 19
  20. 20. Visual Features: DNN vs. AVF • Deep Neural Networks (DNN): we used an AlexNet DNN (pre-trained with ImageNet ILSVRC 2012 dataset) (Krizhevsky et al, 2012) • We map each artwork image to its corresponding latent vector of 4,096 dimensions obtained at the fc6 layer of the AlexNet network.
  21. 21. Visual Features: AVF • Average brightness, • Saturation, • Sharpness, • Entropy, • RGB-contrast, • Colorfulness, • Naturalness Jose San Pedro and Stefan Siersdorfer. 2009. Ranking andClassifying Attractiveness of Photos in Folksonomies. In Proceedings of the 18thInternational Conference on World Wide Web (WWW’09). March 17th 2019 Dominguez et al ~ ACM IUI 2019 21
  22. 22. UGallery: Making Recommendations • Scoring items based on cosine similarity between user model and item model:
  23. 23. Generating Explanations March 17th 2019 Dominguez et al ~ ACM IUI 2019 23 Friedrich, G., & Zanker, M. (2011). A taxonomy for generating explanations in recommender systems. AI Magazine, 32(3), 90-98.
  24. 24. White-box Explanation (AVF) March 17th 2019 Dominguez et al ~ ACM IUI 2019 24
  25. 25. Black-box explanation (DNN and AVF) March 17th 2019 Dominguez et al ~ ACM IUI 2019 25
  26. 26. Study Procedure March 17th 2019 Dominguez et al ~ ACM IUI 2019 26
  27. 27. Study Procedure March 17th 2019 Dominguez et al ~ ACM IUI 2019 27
  28. 28. Study Procedure March 17th 2019 Dominguez et al ~ ACM IUI 2019 28
  29. 29. Preference Elicitation • We collect user preferences from a Pinterest-like interface March 17th 2019 Dominguez et al ~ ACM IUI 2019 29
  30. 30. Study Procedure March 17th 2019 Dominguez et al ~ ACM IUI 2019 30
  31. 31. Interface 1: no explanation, no transparency March 17th 2019 Dominguez et al ~ ACM IUI 2019 31
  32. 32. Study Procedure March 17th 2019 Dominguez et al ~ ACM IUI 2019 32
  33. 33. Interface 2: explainable, no transparency March 17th 2019 Dominguez et al ~ ACM IUI 2019 33
  34. 34. Study Procedure March 17th 2019 Dominguez et al ~ ACM IUI 2019 34
  35. 35. Interface 3: explainable & transparent March 17th 2019 Dominguez et al ~ ACM IUI 2019 35
  36. 36. Study Procedure March 17th 2019 Dominguez et al ~ ACM IUI 2019 36
  37. 37. Evaluation & Results • Post-algorithm survey aspects (agreement1-100) – Explainable:I understood why the art images were recommended to me. – Relevance:The art images recommended matched my interests. – Diverse: The art images recommended were diverse. – Interface Satisfaction: Overall, I am satisfied with the recommender interface. – Use Again: I would use this recommender system again for finding art images in the future. – Trust: I trusted the recommendations made. • Finally, NASA TLX (cognitive effort) March 17th 2019 Dominguez et al ~ ACM IUI 2019 37
  38. 38. Demo March 17th 2019 Dominguez et al ~ ACM IUI 2019 38
  39. 39. Evaluation & Results Study on Amazon Mechanical Turk: ● 121 valid users completed correctly the study. ● Task took them around 10 minutes to complete. ● ~56% female, 44% male. ● 80% attended to 1 or more art classes at high school level or above. ● 80% visited museums or art galleries at least once a year. March 17th 2019 Dominguez et al ~ ACM IUI 2019 39
  40. 40. Results March 17th 2019 Dominguez et al ~ ACM IUI 2019 40 Interface 1: UI without explanation Interface 2: UI with example-based explanation Interface 3: UI with transparent explanation (AVF)
  41. 41. Results March 17th 2019 Dominguez et al ~ ACM IUI 2019 41 7 dimensions evaluated, for DNN and AVF (scale 1-100): Perception of: - Explainability - Relevance - Diversity - Satisfaction w/UI - Intention of use - Trust on RecSys - Avg. Rating
  42. 42. Results • Result 1: DNN better or equal than AVF except on diversity March 17th 2019 Dominguez et al ~ ACM IUI 2019 42 - AVF is perceived significantly more diverse than DNN excepting when AVF uses transparent and explainable recommendations.
  43. 43. Results • Result 2: Explainable interfaces increase perception of explainability. March 17th 2019 Dominguez et al ~ ACM IUI 2019 - Result expected: people perceive the system as more explainable using the explainable interfaces than non explainable. 43
  44. 44. Results • Result 2: Explainable interfaces increase perception of explainability. March 17th 2019 Dominguez et al ~ ACM IUI 2019 - Result expected: people perceive the system as more explainable using the explainable interfaces than non explainable. - But this perception is significantly higher using DNN visual features than AVF. 44
  45. 45. Evaluation & Results • Result 2: Perception of relevance changes just by adding explanations -> User Interface really matters!! March 17th 2019 Dominguez et al ~ ACM IUI 2019 - Algorithm is the same (DNN), but by adding explanations people perceive recommendations as more relevant, - Result is significant only with DNN. 45
  46. 46. Evaluation & Results • Result 3: No difference in Trust between DNN and AVF in I1 (without explanations) March 17th 2019 Dominguez et al ~ ACM IUI 2019 46 - The difference in Trust between DNN and AVF becomes significant only when using explainable interfaces.
  47. 47. Evaluation & Results • Result 4: Without explanations, No difference between DNN and AVF upon intention of use and satisfaction with interface. March 17th 2019 Dominguez et al ~ ACM IUI 2019 47 - The difference on Satisfaction and Intention of use between DNN and AVF becomes significant only when using explainable interfaces.
  48. 48. General Model: Knijnenburg Framework March 17th 2019 Dominguez et al ~ ACM IUI 2019 48 Knijnenburg,B. P., & Willemsen,M. C. (2015).Evaluatingrecommender systemswith user experiments.In RecommenderSystems Handbook (pp.309-352).Springer,Boston,MA.
  49. 49. Visual Art RecSys Explanation Model March 17th 2019 Dominguez et al ~ ACM IUI 2019 49
  50. 50. Visual Art RecSys Explanation Model March 17th 2019 Dominguez et al ~ ACM IUI 2019 50 - Compared to the baseline interface I1, the interface I2 (β=0.551) and interface I3 (β=0.556) have a positive effect on Understandability. - The model explains 23.3% of the variance of Understandability.
  51. 51. Visual Art RecSys Explanation Model March 17th 2019 Dominguez et al ~ ACM IUI 2019 51 - Compared to the baseline AVF, DNN features have an strong positive effect on understandability (β=0.710), as well as on ratings (β=0.710).
  52. 52. Visual Art RecSys Explanation Model March 17th 2019 Dominguez et al ~ ACM IUI 2019 52 - Perceived Effort (Insecurity, rush, mental demand) has a significant negative effect (β=-0.283) on understandability.
  53. 53. Visual Art RecSys Explanation Model March 17th 2019 Dominguez et al ~ ACM IUI 2019 53 - Satisfaction is strongly affected by the Trust on the system (+), as well as by Understandability (+), Time (+) and Experience with the domain (+). - Trust is affected by Understandability (+) and by effort (-)
  54. 54. Conclusion • DNN features performed better than AVF features, probably because AlexNet is able to capture more complex patterns. Results confirm previous work by Messina et al. (2018). • Providing a UI with explanations has a big effect on several dimensions evaluated. This study provides further evidence upon the importance of user interfaces and not only algorithms as important effects on UX with RecSys. • Results indicate that explainability has a positive effect on UX with a RecSys, but is not always well perceived with additional transparency. March 17th 2019 Dominguez et al ~ ACM IUI 2019 54
  55. 55. Future Work • (limitation) If additional transparency is provided in an explanation, make sure the explanation is actually being understood. • Develop a full end-to-end Deep Learning art recommendation model, rather than using pre-trained embeddings and memory- based content-based recommendation. • Test recent advances in XAI and visualization of CNNs into the recommendation framework. March 17th 2019 Dominguez et al ~ ACM IUI 2019 55
  56. 56. Acknowledgment • The were funded by PUC Chile, Conicyt, Fondecyt grant 11150783, as well as by the Millennium Institute for Foundational Research on Data (IMFD). March 17th 2019 Dominguez et al ~ ACM IUI 2019 56
  57. 57. THANKS! Denis Parra Assistant Professor Pontificia Universidad Católicade Chile dparras@uc.cl
  58. 58. References ● Vicente Dominguez, Pablo Messina, Denis Parra, Domingo Mery, Christoph Trattner, and Alvaro Soto. 2017. Comparing Neural and Attractiveness-based Visual Features for Artwork Recommendation. In Proceedings of the Workshop on Deep Learning for Recommender Systems, co-located at RecSys 2017. ● Ruining He, Chen Fang, Zhaowen Wang, and Julian McAuley. 2016. Vista: A Visually, Socially, and Temporally-aware Model for Artistic Recommendation. In Proceedings of the 10th ACM Conference on Recommender Systems (RecSys ’16). ● Alex Krizhevsky, Ilya Sutskever, and Geo rey E Hinton. 2012. Imagenet classi cation with deep convolutional neural networks. In Advances in neural information processing systems. ● Pablo Messina, Vicente Dominguez, , Denis Parra, Christoph Trattner, and Alvaro Soto. 2018. Content-Based Artwork Recommendation: Integrating Painting Metadata with Neural and Manually-Engineered Visual Features. User Modeling and User-Adapted Interaction (2018). ● Jose San Pedro and Stefan Siersdorfer. 2009. Ranking and Classifying Attractiveness of Photos in Folksonomies. In Proceedings of the 18th International Conference on World Wide Web (WWW ’09). March 17th 2019 Dominguez et al ~ ACM IUI 2019 58

×