Presented at RecSys2012, Dublin. Any questions or comments, email me at marcotcr@gmail.com

1. Pareto-Efficient Hybridization for
Multi-Objective Recommender
Systems
Marco Tulio Ribeiro 1,2 Anisio Lacerda 1,2
Adriano Veloso 1 Nivio Ziviani 1,2
1 2
Universidade Federal de Minas Gerais Zunnit Technologies
Computer Science Department Belo Horizonte, Brazil
Belo Horizonte, Brazil
ACM Recommender Systems 2012, Dublin, Ireland
September 10th, 2012
1

2. Pareto Efficient Hybridization for
Multi-Objective Recommender Systems
2

3. Pareto Efficient Hybridization for
Multi-Objective Recommender Systems
Multi-Objective:
2

4. Pareto Efficient Hybridization for
Multi-Objective Recommender Systems
Multi-Objective:
Accuracy
Novelty
Diversity
2

5. Pareto Efficient Hybridization for
Multi-Objective Recommender Systems
Multi-Objective:
Accuracy
Novelty
Diversity
Hybridization:
Different algorithms have different strengths
2

6. Pareto Efficient Hybridization for
Multi-Objective Recommender Systems
Multi-Objective:
Accuracy
Novelty
Diversity
Hybridization:
Different algorithms have different strengths
Pareto Efficient:
In a moment
2

7. What’s a Good Recommendation?
“Good” is a multifaceted concept
3

8. What’s a Good Recommendation?
“Good” is a multifaceted concept
Are novel recommendations good
recommendations?
3

9. Is Novelty Good?
3

10. Is Novelty Good?
3

11. What’s a Good Recommendation?
“Good” is a multifaceted concept
Are novel recommendations good
recommendations?
Are accurate recommendations good
recommendations?
3

12. Is Accuracy Good?
3

13. Is Accuracy Good?
3

14. What’s a Good Recommendation?
“Good” is a multifaceted concept
Are novel recommendations good
recommendations?
Are accurate recommendations good
recommendations?
Are diverse recommendations good
recommendations?
3

15. Is Diversity Good?
3

16. Is Diversity Good?
3

17. Our Work
The challenge:
Combining multiple algorithms
4

18. Our Work
The challenge:
Combining multiple algorithms
Contributions:
Domain and algorithm-independent hybrid
4

19. Our Work
The challenge:
Combining multiple algorithms
Contributions:
Domain and algorithm-independent hybrid
Multi-objective in terms of accuracy, novelty
and diversity.
4

20. Our Work
The challenge:
Combining multiple algorithms
Contributions:
Domain and algorithm-independent hybrid
Multi-objective in terms of accuracy, novelty
and diversity.
Adjustable compromise
4

21. Weighted Aggregation
Combine the algorithms using standard
weighted aggregation
5

22. Weighted Aggregation
Combine the algorithms using standard
weighted aggregation
Problem: finding the vector of weights W
5

23. Weighted Aggregation
Combine the algorithms using standard
weighted aggregation
Problem: finding the vector of weights W
Example:
W = [SVD: 2.3, TopPop: −5, ItemKNN : 1]
5

24. Weighted Aggregation
Combine the algorithms using standard
weighted aggregation
Problem: finding the vector of weights W
Example:
W = [SVD: 2.3, TopPop: −5, ItemKNN : 1]
Easy to add or remove algorithms
5

25. Evolutionary Algorithms
A population is created with a group of
random individuals
6

26. Evolutionary Algorithms
A population is created with a group of
random individuals
For each generation:
The individuals of the population are
evaluated (cross validation)
The best individuals are combined, mutated
or kept
6

27. Evolutionary Algorithms
A population is created with a group of
random individuals
For each generation:
The individuals of the population are
evaluated (cross validation)
The best individuals are combined, mutated
or kept
Good for search spaces where little is
known
6

28. Evolutionary Algorithms
A population is created with a group of
random individuals
For each generation:
The individuals of the population are
evaluated (cross validation)
The best individuals are combined, mutated
or kept
Good for search spaces where little is
known
Domain and algorithm-independent
6

29. SPEA2
Strength Pareto Evolutionary Algorithm
[Zitzler, Laumanns and Thiele]
7

30. SPEA2
Strength Pareto Evolutionary Algorithm
[Zitzler, Laumanns and Thiele]
Multi-Objective Evolutionary Algorithm
7

31. SPEA2
7

32. SPEA2
Strength Pareto Evolutionary Algorithm
[Zitzler, Laumanns and Thiele]
Multi-Objective Evolutionary Algorithm
Uses the Pareto Dominance concept
7

33. Pareto Dominance
7

34. Pareto Dominance
7

35. Pareto Dominance
7

36. Pareto Dominance
7

37. SPEA2
Strength Pareto Evolutionary Algorithm
[Zitzler, Laumanns and Thiele]
Multi-Objective Evolutionary Algorithm
Uses the Pareto Dominance concept
Returns a Pareto Frontier
7

38. Pareto Frontier
7

39. SPEA2
Strength Pareto Evolutionary Algorithm
[Zitzler, Laumanns and Thiele]
Multi-Objective Evolutionary Algorithm
Uses the Pareto Dominance concept
Returns a Pareto Frontier
O(M 2logM ), but performed offline
7

40. Adjusting the System Priority
The recommender system may desire to
adjust the compromise
8

41. Adjusting the System Priority
The recommender system may desire to
adjust the compromise
We do not return a single solution, but the
Pareto Frontier
8

42. Adjusting the System Priority
The recommender system may desire to
adjust the compromise
We do not return a single solution, but the
Pareto Frontier
Given the priority of each objective, we
choose one individual from the frontier
8

43. Adjusting the System Priority
8

44. Evaluation Methodology
Task: Top-N Item Recommendation
9

45. Evaluation Methodology
Task: Top-N Item Recommendation
Evaluation methodology similar to
[Cremonesi, Koren and Turrin, RecSys 2010]
9

46. Evaluation Methodology
Task: Top-N Item Recommendation
Evaluation methodology similar to
[Cremonesi, Koren and Turrin, RecSys 2010]
With novelty and diversity from
[Vargas and Castells, RecSys 2011]
9

47. Datasets
Movielens Last.fm
Recommends movies music
Users 6,040 992
Content 3,883 movies 176,948 artists
Ratings/Feedback 1,000,209 19,150,868
Feedback explicit implicit
Table: Summary of Datasets
10

48. Recommendation Algorithms
PureSVD (50 and 150 factors)
[Cremonesi, Koren and Turrin, RecSys 2010]
11

49. Recommendation Algorithms
PureSVD (50 and 150 factors)
[Cremonesi, Koren and Turrin, RecSys 2010]
KNNs: Item and User-based
11

50. Recommendation Algorithms
PureSVD (50 and 150 factors)
[Cremonesi, Koren and Turrin, RecSys 2010]
KNNs: Item and User-based
Most Popular
11

51. Recommendation Algorithms
PureSVD (50 and 150 factors)
[Cremonesi, Koren and Turrin, RecSys 2010]
KNNs: Item and User-based
Most Popular
WRMF
[Hu et al, ICDM 2008, Pan et al ICDM 2008]
11

52. Recommendation Algorithms
PureSVD (50 and 150 factors)
[Cremonesi, Koren and Turrin, RecSys 2010]
KNNs: Item and User-based
Most Popular
WRMF
[Hu et al, ICDM 2008, Pan et al ICDM 2008]
Content-based:
Item Attribute KNN (movielens only)
User Attribute KNN
11

53. Hybrid Baselines
Borda Count
12

54. Hybrid Baselines
Borda Count
STREAM (stacking-based approach)
[Bao, Bergman and Thompson, RecSys 2009]
12

55. Hybrid Baselines
Borda Count
STREAM (stacking-based approach)
[Bao, Bergman and Thompson, RecSys 2009]
Weighted aggregation with equal weights
12

56. Some of Our Solutions - Movielens
PO-acc:
PO-acc2:
PO-nov:
PO-div:
13

57. 14

58. 15

59. 16

60. Some of Our Solutions - Last.fm
PO-acc:
PO-nov:
PO-div:
17

61. 18

62. 19

63. 20

64. Conclusions
A multi-objective hybridization technique for
combining recommendation algorithms
21

65. Conclusions
A multi-objective hybridization technique for
combining recommendation algorithms
“Tune” the system to different priority needs
21

66. Conclusions
A multi-objective hybridization technique for
combining recommendation algorithms
“Tune” the system to different priority needs
Highly reproducible experiments:
Public datasets
Open-source implementations
(MyMediaLite, DEAP)
21

67. Conclusions
A multi-objective hybridization technique for
combining recommendation algorithms
“Tune” the system to different priority needs
Highly reproducible experiments:
Public datasets
Open-source implementations
(MyMediaLite, DEAP)
Competitive with the best algorithms
according to each objective
21

68. Future Work
Test these assumptions using online
AB-testing, in real world E-commerce
websites
22

69. Future Work
Test these assumptions using online
AB-testing, in real world E-commerce
websites
Try maximizing other objectives:
profit, stock diversity, etc
22

70. Future Work
Test these assumptions using online
AB-testing, in real world E-commerce
websites
Try maximizing other objectives:
profit, stock diversity, etc
Figuring out how often the weights need to
be re-adjusted
22

71. Pareto-Efficient Hybridization for
Multi-Objective Recommender
Systems
Marco Tulio Ribeiro 1,2 Anisio Lacerda 1,2
Adriano Veloso 1 Nivio Ziviani 1,2
1 2
Universidade Federal de Minas Gerais Zunnit Technologies
Computer Science Department Belo Horizonte, Brazil
Belo Horizonte, Brazil
ACM Recommender Systems 2012, Dublin, Ireland
September 10th, 2012
23