Presentation at 20171025 reading group

1. Retweet Prediction with Attention-
based Deep Neural Network
#CIKM2016
Authors: Qi Zhang, Yeyun Gong, Jindou Wu, Haoran Huang, Xuanjing Huang
Reading group: 25/10/2017
Presenter: Guangyuan Piao (Unit for Social Semantics)
Mentor: Subhasis Thakur | Supervisor: John G. Breslin

2. Agenda
• Background & Related Work
• Proposed Approach
• Experimental Setup & Results
• Conclusions
• Summary
2

3. Background
3
• easy real-time information sharing
• 1 billion unique visits / month for Twitter

4. Background – Retweeting Behavior
4

5. Background – Retweeting Behavior
5
• key mechanism for spreading information
• can help information spreading prediction, popularity prediction etc.

6. (Some) Related Work
6
• Retweeting behavior
• study of a number of features for retweetability of tweets [Suh et al., SocialCom’10]
• feature-aware factorization model [Feng et al, WSDM’13]
• considering information about user, tweet, and author
• who will retweet me? [Luo et al., SIGIR’13]
• using learning-to-rank framework
• non-parametric statistical models [Zhang et al. AAAI’15]
• combining structural, textual & temporal info.

7. (Some) Related Work
7
• Retweeting behavior
• study of a number of features for retweetability of tweets [Suh et al., SocialCom’10]
• feature-aware factorization model [Feng et al, WSDM’13]
• considering information about user, tweet, and author
• who will retweet me? [Luo et al., SIGIR’13]
• using learning-to-rank framework
• non-parametric statistical models [Zhang et al. AAAI’15]
• combining structural, textual & temporal info.
feature engineering is required

8. (Some) Related Work
8
• Convolutional Neural Network (CNN)
• image recognition
• video processing
• natural language processing
• Attention-based Neural Network
• machine translation
• speech recognition
• visual object classification

9. Proposed Approach – Variants of CNN approach
9
words of a tweet
• Vu: user embedding vector
• Vp: tweet embedding vector

10. Proposed Approach – Variants of CNN approach
10
• Vu: user embedding vector
• Vp: tweet embedding vector
• Va: author embedding vector

11. Proposed Approach
11
• Modeling User Interests based on Tweet History [t1, t2 … tm]
• clustering m tweets of each user into n groups using K-means
• using the central tweet of each group as an interest of user
• user interest profile [t1, t2 … tn]

12. • Modeling User Interests based on Tweet History [t1, t2 … tm]
• clustering m tweets of each user into n groups
• using the central tweet of each group as an interest of user
• user interest profile [t1, t2 … tn]
• apply CNN for each tweet to obtain
tweet embeddings
Proposed Approach
12

13. Proposed Approach
13
• Attention
• Folding
the value in the i-th position of the embedding of the j-th attention interests

14. Proposed Approach
14

15. Proposed Approach
15
• Vu: user embedding vector
• Vi: user interest embedding vector
• S: similarity(user interest vector, tweet vector)
• Vp: tweet embedding vector
• Va: author embedding vector

16. Experiment Setup
16
• Twitter Dataset
• 75% (training, 10% for validation), 25% (test)
• Evaluation Metrics
• precision
• recall
• F1-score

17. Experiment Setup
17
• Model Parameters
• dropout rate: 0.5
• window size: (1, 2)
• feature maps num.: 100
• L2 constraint: 3
• mini-batch size: 40
• cluster number: 5
• word vector: word2vec trained based on Google News
• user & author vector dimensions: 300 (the same as word embedding)

18. Experiment Setup
18
• Compared Methods
• Random: random decision
• Ave-SVM, Sum-SVM: average, sum of word vectors for tweet vectors
• ASC-HDP: non-parametric statistical models [Zhang et al. AAAI’15]
• CNN, U-CNN, UA-CNN
• SUA-ACNN: with attention

19. Experimental Results
19

20. Experimental Results
20

21. Conclusions
21
• Proposed a novel attention-based deep neural network
• that can perform better than state-of-the-art methods for retweet prediction
• user, author embeddings, the similarity score and the user’s attention interests can
each significantly improve the performance
• the integration of these components provides the best performance

22. 22
Guangyuan Piao
e-mail: guangyuan.piao@insight-centre.org
twitter: https://twitter.com/parklize
slideshare: http://www.slideshare.net/parklize