Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Inria, SequeL
1
Inria, Sequel – Meetup Juin 2016
Deep Learning
Collaborative
Filtering
and
Inria, SequeL
2
Collaborative filtering
Bob
Inria, SequeL
3
Bob
Collaborative filtering
Tim
Inria, SequeL
4
Bob
Tim
Collaborative filtering
Inria, SequeL
5
Collaborative filtering
Bob
Tim
Inria, SequeL
6
Collaborative filtering
Bob
Tim
Inria, SequeL
7
Collaborative filtering
Bob
Tim
Inria, SequeL
8
Goal is to predict the rating Tim
would have given to Jurassic Park
Collaborative filtering
Tim
Inria, SequeL
9
Collaborative filtering
Inria, SequeL
10
Goal : Predict the missing rating
Collaborative filtering
Bob ? 3 5 ?
Ana ? 4 ? 5
Elain 5 ? 4 ?
Sulman 2 ...
Inria, SequeL
11
Goal : Predict the missing rating
Bob 4 3 5 1
Ana 2 4 2 5
Elain 5 4 4 3
Sulman 2 3 1 4
Collaborative filt...
Inria, SequeL
12
Goal : Predict the missing rating
Bob 4 3 5 1
Ana 2 4 2 5
Elain 5 4 4 3
Sulman 2 3 1 4
To be recommended
...
Inria, SequeL
13
Bob ? 3 5 ?
Ana ? 4 ? 5
Elain 5 ? 4 ?
Sulman 2 ? 1 4
Score: Average|Predicted rating – real rating|
Colla...
Inria, SequeL
14
Bob ? 3 5 ?
Ana ? 4 ? 5
Elain 5 ? 4 ?
Sulman 2 ? 1 4
Score = Average |Predicted rating – real rating|2
Co...
Inria, SequeL
15
Neural Networks
Inria, SequeL
16
Input
Neural Networks
Inria, SequeL
17
Forward
Backward
The model is updated to fit the
target
The error is backpropagated into
the model
Neural...
Inria, SequeL
18
Neural Networks
Inria, SequeL
19
RUsers
Items v
u
Neural Networks
Inria, SequeL
20
RUsers
Items v
u
Neural Networks
Inria, SequeL
21
RUsers
Items v
u
Neural Networks
Inria, SequeL
22
? 3 4 ? 1
4.9 2.8 4 .1 2.0 1.1
𝑅
user
𝑅
i
t
e
m
U-CFN V-CFN
Neural Networks
Inria, SequeL
23
? 3 4 ? 1
4.9 2.8 4 .1 2.0 1.1
inputs
Dimension
reduction
Input reconstruction
5 3 4 2 1
4.9 2.8 4 .1 2.0...
Inria, SequeL
24
? 3 4 ? 1
4.9 2.8 4 .1 2.0 1.1
3 4 1
2.8 4 .1 1.1
Training stepTesting step
The model
Inria, SequeL
25
1 ? 2 ? 1
1.2 4.9 1.1
1 0 2 0 1
1.2 2.1 4.9 2.0 1.1
Forward
The model
Inria, SequeL
26
1 ? 2 ? 1
1.2 4.9 1.1
1 0 2 0 1
0.2 0 -0.1 0 0.1
Backward
The model
Inria, SequeL
27
β*0.2 0 α*1.1 0 β*1.1
1 0 2 0 1
1 0 0 0 1
Forward
1.2 2.3 3.1 4.2 1.1
Backward
1 ? 2 ? 1
True input
0.2 0...
Inria, SequeL
28
β*0.2 0 α*1.1 0 β*1.1
1 0 2 0 1
1 0 0 0 1
Forward
1.2 2.3 3.1 4.2 1.1
Backward
1 ? 2 ? 1
True input
0.2 0...
Inria, SequeL
29
The model
300.000 to 50.000 entries
Real
life
Inria, SequeL
30
MovieLens :
- 10 million ratings : 72.000 users / 10.000 movies
- 20 million ratings : 138.000 users / 28...
Inria, SequeL
31
Quadratic error
MovieLens-10M MovieLens-20M
BPMF 0.8213 0.8123
ALS-WR 0.7830 0.7746
LLORMA 0.7949 0.7843
...
Inria, SequeL
32
Singular value decomposition (SVD)
Other algorithms:
●Alternating Least Square Weighted Lambda-Regulariza...
Inria, SequeL
33
RUsers
Items v
u
z = W3 𝐚𝐜𝐭𝐢𝐯𝐚𝐭𝐢𝐨𝐧2
ri = Vui
activation
Link with matrix factorization (optional)
Inria, SequeL
34
ui
V
Link with matrix factorization (optional)
activation
ri = Vui
RUsers
Items v
u
CFN computes a Non-Li...
Inria, SequeL
35
Discussion
Why nobody tried it? Image/sounds
are dense
Few works
about sparsity
No tools!
Inria, SequeL
36
Discussion
Torch framework
Reproduce results
Inria, SequeL
37
? 3 4 ? 1
4.9 2.8 4 .1 2.0 1.1
Actor :
Dicaprio
Genre:
action
Year:
2010
Adding external information
Inria, SequeL
38
4.9 2.8 4 .1 2.0 1.1
Content-based Filtering
Extensions
Inria, SequeL
39
Thank you for listening!
Inria, SequeL
40
Input reconstruction
5 ? 4 ? 1
4.9 2.8 4 .1 2.0 1.1
Encoder
Decoder
Autoencoders:
Matrix factorization
z ...
Prochain SlideShare
Chargement dans…5
×

Using Neural Networks to predict user ratings

2 134 vues

Publié le

Talk given by Florian Strub during the RecsysFR meetup on June 22nd 2016.

Publié dans : Internet

Using Neural Networks to predict user ratings

  1. 1. Inria, SequeL 1 Inria, Sequel – Meetup Juin 2016 Deep Learning Collaborative Filtering and
  2. 2. Inria, SequeL 2 Collaborative filtering Bob
  3. 3. Inria, SequeL 3 Bob Collaborative filtering Tim
  4. 4. Inria, SequeL 4 Bob Tim Collaborative filtering
  5. 5. Inria, SequeL 5 Collaborative filtering Bob Tim
  6. 6. Inria, SequeL 6 Collaborative filtering Bob Tim
  7. 7. Inria, SequeL 7 Collaborative filtering Bob Tim
  8. 8. Inria, SequeL 8 Goal is to predict the rating Tim would have given to Jurassic Park Collaborative filtering Tim
  9. 9. Inria, SequeL 9 Collaborative filtering
  10. 10. Inria, SequeL 10 Goal : Predict the missing rating Collaborative filtering Bob ? 3 5 ? Ana ? 4 ? 5 Elain 5 ? 4 ? Sulman 2 ? 1 4
  11. 11. Inria, SequeL 11 Goal : Predict the missing rating Bob 4 3 5 1 Ana 2 4 2 5 Elain 5 4 4 3 Sulman 2 3 1 4 Collaborative filtering
  12. 12. Inria, SequeL 12 Goal : Predict the missing rating Bob 4 3 5 1 Ana 2 4 2 5 Elain 5 4 4 3 Sulman 2 3 1 4 To be recommended Collaborative filtering
  13. 13. Inria, SequeL 13 Bob ? 3 5 ? Ana ? 4 ? 5 Elain 5 ? 4 ? Sulman 2 ? 1 4 Score: Average|Predicted rating – real rating| Collaborative filtering
  14. 14. Inria, SequeL 14 Bob ? 3 5 ? Ana ? 4 ? 5 Elain 5 ? 4 ? Sulman 2 ? 1 4 Score = Average |Predicted rating – real rating|2 Collaborative filtering
  15. 15. Inria, SequeL 15 Neural Networks
  16. 16. Inria, SequeL 16 Input Neural Networks
  17. 17. Inria, SequeL 17 Forward Backward The model is updated to fit the target The error is backpropagated into the model Neural Networks
  18. 18. Inria, SequeL 18 Neural Networks
  19. 19. Inria, SequeL 19 RUsers Items v u Neural Networks
  20. 20. Inria, SequeL 20 RUsers Items v u Neural Networks
  21. 21. Inria, SequeL 21 RUsers Items v u Neural Networks
  22. 22. Inria, SequeL 22 ? 3 4 ? 1 4.9 2.8 4 .1 2.0 1.1 𝑅 user 𝑅 i t e m U-CFN V-CFN Neural Networks
  23. 23. Inria, SequeL 23 ? 3 4 ? 1 4.9 2.8 4 .1 2.0 1.1 inputs Dimension reduction Input reconstruction 5 3 4 2 1 4.9 2.8 4 .1 2.0 1.1 The model
  24. 24. Inria, SequeL 24 ? 3 4 ? 1 4.9 2.8 4 .1 2.0 1.1 3 4 1 2.8 4 .1 1.1 Training stepTesting step The model
  25. 25. Inria, SequeL 25 1 ? 2 ? 1 1.2 4.9 1.1 1 0 2 0 1 1.2 2.1 4.9 2.0 1.1 Forward The model
  26. 26. Inria, SequeL 26 1 ? 2 ? 1 1.2 4.9 1.1 1 0 2 0 1 0.2 0 -0.1 0 0.1 Backward The model
  27. 27. Inria, SequeL 27 β*0.2 0 α*1.1 0 β*1.1 1 0 2 0 1 1 0 0 0 1 Forward 1.2 2.3 3.1 4.2 1.1 Backward 1 ? 2 ? 1 True input 0.2 0 1.1 0 0.1 error 0.2 ? 1.1 ? 0.1 α = Prediction = supervized β = Reconstruction = unsupervized The model 1 ? 2 ? 1
  28. 28. Inria, SequeL 28 β*0.2 0 α*1.1 0 β*1.1 1 0 2 0 1 1 0 0 0 1 Forward 1.2 2.3 3.1 4.2 1.1 Backward 1 ? 2 ? 1 True input 0.2 0 1.1 0 0.1 error 0.2 ? 1.1 ? 0.1 α = Prediction = supervized β = Reconstruction = unsupervized The model 1 ? 2 ? 1
  29. 29. Inria, SequeL 29 The model 300.000 to 50.000 entries Real life
  30. 30. Inria, SequeL 30 MovieLens : - 10 million ratings : 72.000 users / 10.000 movies - 20 million ratings : 138.000 users / 28.000 movies Results
  31. 31. Inria, SequeL 31 Quadratic error MovieLens-10M MovieLens-20M BPMF 0.8213 0.8123 ALS-WR 0.7830 0.7746 LLORMA 0.7949 0.7843 U-CFN 0.7767 0.7663 V-CFN 0.7767 0.7663 RMSE for train/test 90/10 BPMF : rank=10 ALS-WE : rank = 200 LLORMA : rank = 20, anchor point = 30 A. Mnih and R. Salakhutdinov, “Probabilistic matrix factorization,” in Advances in neural information processing systems, 2007, pp. 1257– 1264. B. J. Lee, S. Kim, G. Lebanon, and Y. Singerm, “Local low-rank matrix approximation,” in Proc. of ICML’13, 2013, pp. 82–90. C. Y.Zhou,D.Wilkinson,R.Schreiber,andR.Pan,“Large-scaleparallel collaborative filtering for the netflix prize,” in Algorithmic Aspects in Information and Management. Springer, 2008, pp. 337–348. Results
  32. 32. Inria, SequeL 32 Singular value decomposition (SVD) Other algorithms: ●Alternating Least Square Weighted Lambda-Regularization (ALS-WR) ●Probabilistic Matrix Factorization (PMF, BPMF, NLPMF) ●Local Low Rank Matrix Approximation (LLORMA) RUsers Items v u Link with matrix factorization (optional)
  33. 33. Inria, SequeL 33 RUsers Items v u z = W3 𝐚𝐜𝐭𝐢𝐯𝐚𝐭𝐢𝐨𝐧2 ri = Vui activation Link with matrix factorization (optional)
  34. 34. Inria, SequeL 34 ui V Link with matrix factorization (optional) activation ri = Vui RUsers Items v u CFN computes a Non-Linear Matrix Factorization
  35. 35. Inria, SequeL 35 Discussion Why nobody tried it? Image/sounds are dense Few works about sparsity No tools!
  36. 36. Inria, SequeL 36 Discussion Torch framework Reproduce results
  37. 37. Inria, SequeL 37 ? 3 4 ? 1 4.9 2.8 4 .1 2.0 1.1 Actor : Dicaprio Genre: action Year: 2010 Adding external information
  38. 38. Inria, SequeL 38 4.9 2.8 4 .1 2.0 1.1 Content-based Filtering Extensions
  39. 39. Inria, SequeL 39 Thank you for listening!
  40. 40. Inria, SequeL 40 Input reconstruction 5 ? 4 ? 1 4.9 2.8 4 .1 2.0 1.1 Encoder Decoder Autoencoders: Matrix factorization z = ℎ Wui,𝐝𝐞𝐧𝐬𝐞 ui,𝐬𝐩𝐚𝐫𝐬𝐞 ui,𝐝𝐞𝐧𝐬𝐞 Link with Matrix Factorization

×