This document describes research on using convolutional neural networks (CNNs) for visual sentiment prediction. It presents the methodology which includes fine-tuning a pre-trained CNN model on two datasets labeled for sentiment. Experiments are described that analyze the CNN layer-by-layer, ablate layers, and add new layers. The results show that fine-tuning achieves good performance and deeper layers learn sentiment-related concepts. Future work aims to expand the datasets and analyze network receptive fields.
7. ▷ What? Predict the sentiment that an image provokes to a human
▷ How?
7
Introduction: problem definition
8. ▷ What? Predict the sentiment that an image provokes to a human
▷ How?
8
Introduction: problem definition
9. ▷ What? Predict the sentiment that an image provokes to a human
▷ How? Using Convolutional Neural Networks (CNNs)
9
CNN
Introduction: problem definition
13. Related work: low-level descriptors
13
Siersdorfer, S., Minack, E., Deng, F., & Hare, J. (2010, October).
Analyzing and predicting sentiment of images on the social web. In
Proceedings of the international conference on Multimedia (pp. 715-718).
ACM.
Machajdik, J., & Hanbury, A. (2010, October). Affective image
classification using features inspired by psychology and art theory. In
Proceedings of the international conference on Multimedia (pp. 83-92).
ACM.
14. 14
Borth, D., Ji, R., Chen, T., Breuel, T., & Chang, S. F. (2013, October). Large-scale visual sentiment ontology and detectors using
adjective noun pairs. In Proceedings of the 21st ACM international conference on Multimedia (pp. 223-232). ACM.
Related work: SentiBank
15. Related work: CNNs for sentiment prediction
15
You, Q., Luo, J., Jin, H., & Yang, J. (2015). Robust image sentiment analysis using progressively trained and domain transferred
deep networks. In The Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI).
16. Outline
1. Introduction
2. Related work
3. Methodology and results
a. Convolutional Neural Networks
b. Datasets
c. Experimental setup and results
4. Conclusions
5. Future work
16
18. Outline
1. Introduction
2. Related work
3. Methodology and results
a. Convolutional Neural Networks
b. Datasets
c. Experimental setup and results
4. Conclusions
5. Future work
18
22. Outline
1. Introduction
2. Related work
3. Methodology and results
a. Convolutional Neural Networks
b. Datasets
c. Experimental setup and results
4. Conclusions
5. Future work
22