Deep learning based recommender systems; survey paper review.

1. DM
Deep Learning based Recommender System :
A survey and New Perspectives (1/2)
Zhang Et al. ’17
이현성

2. DMIndex
Introduction
Category of neural network and NN based recommender algorithms
MLP-based recommenders
AE-based recommenders
CNN-based recommenders
RNN-based recommenders
DSSM-based recommendersa

3. DM
Introduction

4. DM
Why should we care about deep learning based RS?
many successes in various application areas in both academia and industry.
Deep Learning techniques enables to exploit complex data
image, tags, text,

5. DM
이 논문이 너무 많은 모델을 다루고 있어서
모델의 성능이나, 자세한 알고리즘은 제외하고
Deep Learning 모델을 사용한 이유
이런 점을 Deep Learning을 이용해 해결하려 했다
같은 Motivation과 Insight만 note하겠습니다.

6. DM
Category of neural network and NN based r
ecommender algorithms

7. DM
Frequently used Deep Learning Techniques
• Multi-Layer Perceptron (MLP)
• Autoencoder (AE)
• Convolutional Neural Network (CNN)
• Recurrent Neural Network (RNN)
• Deep Semantic Similarity Model (DSSM)
• Restricted Boltzmann Machine (RBM)
• Neural Autoregressive Distribution Estimation (NADE)
• Generative Adversarial Network (GAN)

8. DM
Recommender systems as neural network models
1) Models Using Single Deep-Learning Techniques
a) exploit one of mentioned eight techniques
i) Extracting item features from images
ii) Recommendation with explanations
2) Composite Models
a) exploit more than one deep learning techniques
i) GAN using AE as generator, MLP as discriminator.
ii) applying extracted item features using CNN to RNN

9. DM
Integration of Deep learning with Traditional RS
1) Solely Deep-Learning based Models
a) Predicting scores using MLP
1) Coupled Models (Recommender model + Neural network model)
a) AutoEncoder + extracted image features with CNN
i) two model objectives are updated simultaneously (Tightly Coupled models)
ii) two step learning (Loosely Coupled models )

10. DM
Classification of Deep-Learning based Recommender System

11. DM
MLP Based Recommender Systems

12. DMNeural Collaborative Filtering (NCF)
 Motivation
 Previous Matrix Factorization only consi
ders linear relationship between users a
nd items
 𝑦 𝑢𝑖 = 𝑑𝑜𝑡 𝑢, 𝑖
 Solution
 Non-linear Relationship between users
and items
 𝑦 𝑢𝑖 = 𝑓 𝑢, 𝑖 Θ
 Where 𝑓 Θ is multi-layer perceptron.
 User-Item interaction prediction

13. DMWide & Deep Learning
Motivation
 Linear regression is good at
memorization
 Deep MLP is good at generalization,
catching deep features.
Solution
 Integrate linear regression and MLP
구글 플레이스토어에서, 쿼리에 맞
는 application하는 데 사용 중인
알고리즘

14. DMYoutube Video Recommendation (Covington et al.)
Motivation
 There are too many features to tuned
Solution
 MLP works quite well without
exhaustive feature engineering
Youtube에서 personalized video
recommendation을 할 때 사용중
인 알고리즘

15. DM
Autoencoder Based Recommender Systems

16. DM추천 시스템에서 Autoencoder가 주로 사용되는 용
도
Predict interaction between user
and items
Motivation
 Denosing, Reconstruction effects of
Autoencoder will be helpful in
revealing un-observed preferences of
users.
Feature Extraction
Motivation
 Use Autoencoder to learn low-
dimension representation in the
bottleneck layer
Autoencoder는 주로 이 두 가지 방법으로 사용되고, IR과 Recsys
community에서는 상당히 기본적인 방법이 되었다.

17. DMOverview

18. DMAutoencoder as generative model
3
5
4
i1 i2 i3 i4 i5
u1 ? ? 3 4 ?
u2 2 ? 5 ? ?
u3 ? 1 ? ? ?
u4 2 ? 4 ? 3
2.8
4.6
2.3
3.5
Predicted rating 𝑟33
Rating prediction

19. DMAutoencoder as feature extractor
1
0
1
0
animation Horror Action romance
i1
i2 1 0 0 1
i3
i4
0.1
0.8
1
0
0
1
Predicted rating 𝑟33
Low-dimension
representation

20. DM
CNN Based Recommender Systems

21. DMConvolutional Neural Network is
Capable to extract low-dimensional feature from complex data
(image, fixed-size sequences…)
These side information(image) can be used to alleviate cold-start
problem.
Motivation
 CNN-based recommender systems mainly aims to help recommendation quality
using side information

22. DMGeneral CNN based Recommender systems
Motivation
 Images of items contains much
information
Solution
 Use CNN to extract these information
into latent codes.
Image of item i
CNN
Latent code
Item vector of
item i
Output : Item representation
Addition

23. DM
Personalized Image Tag Recommendation
(Nguyen et al.)
Motivation
 CNN can code image into vectors
 User can be embed into vectors
using current ML techniques.
Image에 어울리는 tag를 generate
하기 위해 제안된 알고리즘
Image input
Convolution
Max-pooling User info
Tag Food Holid
ay
Love … Selfie
Score 0.3 0.1 0.5 … 0.8
Image에서 feature를 미리
추출해서 pretrained vector로
사용하는 경우도 많다.

24. DMCNN for Audio Feature Extraction
Motivation
 Allivate cold-start problem using
Content information
Solution
 Learn latent vector representation
using Audio features
Audio Features for item i
Convolution
𝑦𝑖′ 𝑦𝑖
𝑦𝑖 는 WMF model이 만든 item i의 latent
representation

25. DM
RNN Based Recommender Systems

26. DMRecurrent Neural Network
Mainly concerned with sequential behaviors
User/Item evolves with time (temporal dynamics)
Session based recommendations
 rare user information is known
 Sequential behavior become more important than normal cases.
Text understanding
 Like CNN, RNN enables to exploit rich side information(text)

27. DMGRU-Rec (Hidasi et al.)
Motivation
 user’s each click in web-page is
closely related with each others
Solution
 Bidirectional GRU fits to model this
sequential behavior well
Input :
One-
hot
Enco
ded
Vect
or
Emb
eddin
g
Layer
GRU
Layer
GRU
Layer
GRU
Layer
…
Feed
forwa
rd
Layer
Outp
ut :
score
s on
items

28. DMRecurrent Recommender Network (Wu et al.)
Motivation
 Users and items behavior change
over time
Solution
 Models user and item behaviors
using LSTM
 𝒓 𝒖𝒊|𝒕 = 𝒇 𝒖 𝒖, 𝒖 𝒖𝒕, 𝒗𝒊, 𝒗𝒊𝒕

29. DMGRU Multitask Learning (Bansal et al.)
Motivation
 Rich text information will be helpful
Solution
 Use GRU to encode text sequences
(review, description of items)
Text description
of item i
GRU
Latent code
Item vector of
item i
Output : Item representation
Addition
여러 representation을 합칠 때,
의외로 dot이나 elementwise
multiplication보다 단순 addition이
성능이 더 좋은 경우가 많은 것 같다.

30. DM
DSSM Based Recommender Systems

31. DMDeep Structured Semantic Model

32. DMDeep Structured Semantic Model

33. DMTraining DSSM
Query와 relevant한 document의
cosine similarity는 1에 가깝게,
Query와 irrevant한 document의
cosine similarity는 0에 가깝게.

34. DMDeep Structured Semantic Model
Motivation
 User,item 둘 다 side-information을 갖
고 있다.
Solution
 DSSM을 그대로 적용하자.
 DSSM 자체는 similarity measure를
제외하면 Neural Collaborative
filtering, 혹은 MF with side
information의 generalization으로 볼
수 있다.
Query가 아닌 user의
information을 input으로

35. DMFuture research directions
Better understanding of users and items
 딥러닝 모델은 이미지나 텍스트를 잘 모델링한다!(적어도 기존의 방법보다는) 하지만
이러한 모델링은 아직 미숙한 수준이며(Deep learning에서 이용되는 아이디어를 그대
로 가져온 수준), 더 잘 하는 방법이 있을 것이다.
Temporal Dynamics
 Session based Recommendation systems, and recommender with temporal
dynamics are not novel research topics, but they are largely underinvestiagted.

36. DMFuture research directions
Cross-domain / Multi-task learning
 User의 purchase behavior 등을 은행 등에서 활용하려는 시도가 있다.
 한 도메인의 knowledge을, 다른 도메인에 적용하거나,
 여러 도메인의 지식을 동시에 활용했을 시의 성능 향상을 기대할 수 있다.
Novel Evaluation Metrics
 Accuracy, Relevance measures are not good for evaluating recommender.