Context-Aware Recommender System (CARS) models are trained on datasets of context-dependent user preferences (ratings and context information). Since the number of context-dependent preferences increases exponentially with the number of contextual factors, and certain contextual in- formation is still hard to acquire automatically (e.g., the user’s mood or for whom the user is buying the searched item) it is fundamental to identify and acquire those factors that truly influence the user preferences and the ratings. In particular, this ensures that (i) the user effort in specifying contextual information is kept to a minimum, and (ii) the system’s performance is not negatively impacted by irrelevant contextual information. In this paper, we propose a novel method which, unlike existing ones, directly estimates the impact of context on rating predictions and adaptively identifies the contextual factors that are deemed to be useful to be elicited from the users. Our experimental evaluation shows that it compares favourably to various state-of-the-art context selection methods.
2nd Solid Symposium: Solid Pods vs Personal Knowledge Graphs
Parsimonious and Adaptive Contextual Information Acquisition in Recommender Systems
1. IntRS’15 - September 2015, Vienna, Austria
Parsimonious and Adaptive Contextual
Information Acquisition in Recommender
Systems
Matthias Braunhofer1
, Ignacio Fernández-Tobías2
and Francesco Ricci1
1
Free University of Bozen - Bolzano
Piazza Domenicani 3, 39100 Bolzano, Italy
{mbraunhofer,fricci}@unibz.it
2
Universidad Autónoma de Madrid
C / Francisco Tomás y Valiente 11, 28049 Madrid, Spain
ignacio.fernandezt@uam.es
2. IntRS’15 - September 2015, Vienna, Austria
Outline
2
• Introduction
• Related Works
• Selective Context Acquisition
• Experimental Evaluation and Results
• Conclusions and Future Work
3. IntRS’15 - September 2015, Vienna, Austria
Outline
2
• Introduction
• Related Works
• Selective Context Acquisition
• Experimental Evaluation and Results
• Conclusions and
4. IntRS’15 - September 2015, Vienna, Austria
Context-Aware Recommender Systems
• Context-Aware Recommender Systems (CARSs) aim to provide better
recommendations by exploiting contextual information (e.g., weather)
• Rating prediction function is: R: Users x Items x Context → Ratings
3
3 ? 4
2 5 4
? 3 4
1 ? 1
2 5
? 3
3 ? 5
2 5
? 3
5 ? 5
4 5 4
? 3 5
5. IntRS’15 - September 2015, Vienna, Austria
Challenges for CARSs
4
• Identification of contextual factors that influence user preferences and the
decision making process, and hence are worth to be collected from the users
along with their ratings
• Development of a predictive model for predicting the user’s ratings for items
under various contextual situations
• Design of a human-computer interaction layer on top of the predictive model
6. IntRS’15 - September 2015, Vienna, Austria
Challenges for CARSs
4
• Identification of contextual factors that influence user preferences and the
decision making process, and hence are worth to be collected from the users
along with their ratings
• Development of a predictive model for predicting the user’s ratings for items
under various contextual situations
• Design of a human-computer interaction layer on top of the predictive model
7. IntRS’15 - September 2015, Vienna, Austria
Example
STS (South Tyrol Suggests)
5
STS provides context-
aware suggestions for
Places Of Interest (POIs)
in South Tyrol, Italy
8. IntRS’15 - September 2015, Vienna, Austria
Example
STS (South Tyrol Suggests)
5
STS provides context-
aware suggestions for
Places Of Interest (POIs)
in South Tyrol, Italy
9. IntRS’15 - September 2015, Vienna, Austria
Example
STS (South Tyrol Suggests)
5
STS provides context-
aware suggestions for
Places Of Interest (POIs)
in South Tyrol, Italy
10. IntRS’15 - September 2015, Vienna, Austria
Example
STS (South Tyrol Suggests)
5
STS provides context-
aware suggestions for
Places Of Interest (POIs)
in South Tyrol, Italy
11. IntRS’15 - September 2015, Vienna, Austria
Example
STS (South Tyrol Suggests)
5
STS provides context-
aware suggestions for
Places Of Interest (POIs)
in South Tyrol, Italy
12. IntRS’15 - September 2015, Vienna, Austria
Example
STS (South Tyrol Suggests)
5
STS provides context-
aware suggestions for
Places Of Interest (POIs)
in South Tyrol, Italy
13. IntRS’15 - September 2015, Vienna, Austria
Example
STS (South Tyrol Suggests)
5
STS provides context-
aware suggestions for
Places Of Interest (POIs)
in South Tyrol, Italy
14. IntRS’15 - September 2015, Vienna, Austria
Example
STS (South Tyrol Suggests)
5
STS provides context-
aware suggestions for
Places Of Interest (POIs)
in South Tyrol, Italy
15. IntRS’15 - September 2015, Vienna, Austria
Example
STS w/o Selective Context Acquisition
6
Don’t.
All contextual factors are
requested.
16. IntRS’15 - September 2015, Vienna, Austria
Example
STS w/o Selective Context Acquisition
6
Don’t.
All contextual factors are
requested.
17. IntRS’15 - September 2015, Vienna, Austria
Example
STS w/o Selective Context Acquisition
6
Don’t.
All contextual factors are
requested.
18. IntRS’15 - September 2015, Vienna, Austria
Example
STS w/o Selective Context Acquisition
6
Don’t.
All contextual factors are
requested.
19. IntRS’15 - September 2015, Vienna, Austria
Example
STS w/o Selective Context Acquisition
6
Don’t.
All contextual factors are
requested.
20. IntRS’15 - September 2015, Vienna, Austria
Example
STS w/o Selective Context Acquisition
6
Don’t.
All contextual factors are
requested.
21. IntRS’15 - September 2015, Vienna, Austria
Example
STS w/o Selective Context Acquisition
6
Don’t.
All contextual factors are
requested.
22. IntRS’15 - September 2015, Vienna, Austria
Example
STS w/o Selective Context Acquisition
6
Don’t.
All contextual factors are
requested.
23. IntRS’15 - September 2015, Vienna, Austria
Example
STS w/o Selective Context Acquisition
6
Don’t.
All contextual factors are
requested.
24. IntRS’15 - September 2015, Vienna, Austria
Example
STS w/o Selective Context Acquisition
6
Don’t.
All contextual factors are
requested.
25. IntRS’15 - September 2015, Vienna, Austria
Example
STS w/o Selective Context Acquisition
6
Don’t.
All contextual factors are
requested.
26. IntRS’15 - September 2015, Vienna, Austria
Example
STS w/o Selective Context Acquisition
6
Don’t.
All contextual factors are
requested.
27. IntRS’15 - September 2015, Vienna, Austria
Example
STS w/o Selective Context Acquisition
6
Don’t.
All contextual factors are
requested.
28. IntRS’15 - September 2015, Vienna, Austria
Example
STS w/o Selective Context Acquisition
6
Don’t.
All contextual factors are
requested.
29. IntRS’15 - September 2015, Vienna, Austria
Example
STS w/o Selective Context Acquisition
6
Don’t.
All contextual factors are
requested.
30. IntRS’15 - September 2015, Vienna, Austria
Example
STS w/ Selective Context Acquisition
7
Do.
Only relevant contextual
factors are requested.
31. IntRS’15 - September 2015, Vienna, Austria
Example
STS w/ Selective Context Acquisition
7
Do.
Only relevant contextual
factors are requested.
32. IntRS’15 - September 2015, Vienna, Austria
Example
STS w/ Selective Context Acquisition
7
Do.
Only relevant contextual
factors are requested.
33. IntRS’15 - September 2015, Vienna, Austria
Example
STS w/ Selective Context Acquisition
7
Do.
Only relevant contextual
factors are requested.
34. IntRS’15 - September 2015, Vienna, Austria
Outline
8
• Related Works
• Selective Context Acquisition
• Experimental Evaluation and Results
• Conclusions and Future Work
• Introduction
35. IntRS’15 - September 2015, Vienna, Austria
Context Selection
A Priori (i.e., Before Collecting Ratings)
• (Baltrunas et al., 2012): Development of a
web survey where users were requested to
evaluate the influence of contextual
conditions on POI categories
• This allowed to identify the relevant
contextual factors for different POI
categories (using mutual information
statistic)
• Pros: can acquire ratings under relevant
contextual conditions
• Cons: artificial setting; survey requires extra
effort from the user
9
36. IntRS’15 - September 2015, Vienna, Austria
Context Selection
A Posteriori (i.e., After Collecting Ratings)
• (Odić et al., 2013): Provision of several
statistic-based methods for detection of
relevant context, i.e., unalikeability,
entropy, sample variance, χ2
test,
Freeman–Halton test
• Results show a significant difference in
prediction of ratings in context detected as
relevant and the one detected as irrelevant
• Pros: can improve rating prediction
• Cons: still irrelevant context is acquired in
the rating acquisition phase
10
Relevant context Unclassified context
Irrelevant context Baseline predictors
37. IntRS’15 - September 2015, Vienna, Austria
Outline
11
• Related Works
• Selective Context Acquisition
• Experimental Evaluation and Results
• Conclusions and F
• Introduction
38. IntRS’15 - September 2015, Vienna, Austria
Parsimonious & Adaptive Context Acquisition
• Main idea: for each user-item pair (u, i), identify the contextual factors that
when acquired together with u’s rating for i improve most the overall system
• Heuristic: acquire the contextual factors that have the largest impact on
rating prediction
• Example:
12
(Alice, Skiing)
Season
Weather
Temperature
Daytime
Impact
0.000 0.125 0.250 0.375 0.500
39. IntRS’15 - September 2015, Vienna, Austria
Parsimonious & Adaptive Context Acquisition
• Main idea: for each user-item pair (u, i), identify the contextual factors that
when acquired together with u’s rating for i improve most the overall system
• Heuristic: acquire the contextual factors that have the largest impact on
rating prediction
• Example:
12
(Alice, Skiing)
Season
Weather
Temperature
Daytime
Impact
0.000 0.125 0.250 0.375 0.500
How to
quantify this
impact?
40. IntRS’15 - September 2015, Vienna, Austria
CARS Prediction Model
• We use a new variant of Context-Aware Matrix Factorization (CAMF)
(Baltrunas et al., 2011) that treats contextual conditions similarly to either item
or user attributes
• Advantage: allows to capture latent correlations and patterns between a
potentially wide range of knowledge sources ⟹ ideal to derive the usefulness
of contextual factors
13
ˆruic1,...,ck
= (qi + xa
a∈A(i)∪C(i)
∑ )T
⋅(pu + yb
b∈A(u)∪C(u)
∑ )+ ri + bu
qi latent factor vector of item i
A(i) set of conventional item attributes (e.g., genre)
C(i) set of contextual item attributes (e.g., weather)
xa latent factor vector of item attribute a
pu latent factor vector of user u
A(u) set of conventional user attributes (e.g., age)
C(u) set of contextual user attributes (e.g., mood)
yb latent factor vector of user attribute b
ṝi average rating for item i
bu baseline for user u
41. IntRS’15 - September 2015, Vienna, Austria
Largest Deviation
• Computes a personalized relevance score for a contextual factor Cj and a
user-item pair (u, i)
• Given (u, i), it first measures the “impact” of each contextual condition cj ∈ Cj
by calculating the absolute deviation between the rating prediction when the
condition holds (i.e., ȓuicj) and the predicted context-free rating (i.e., ȓui):
where fcj is the normalized frequency of cj
• Finally, it takes the average of these individual scores for the contextual
conditions to yield a single relevance score for the contextual factor Cj
14
ˆwuicj
= fcj
ˆruicj
− ˆrui ,
42. IntRS’15 - September 2015, Vienna, Austria
Illustrative Example
• ȓAlice Skiing Sunny = 5
• ȓAlice Skiing = 3.5
• 20% of ratings are tagged with Sunny (i.e., fSunny = 0.2)
• ŵAlice Skiing Sunny = 0.2⋅|5 - 3.5| = 0.3
15
43. IntRS’15 - September 2015, Vienna, Austria
Outline
16
• Introduction
• Related Works
• Selective Context Acquisition
• Experimental Evaluation and Results
• Conclusions
44. IntRS’15 - September 2015, Vienna, Austria
CoMoDa TripAdvisor
Domain Movies POIs
Rating scale 1-5 1-5
Ratings 2,098 4,147
Users 112 3,916
Items 1,189 569
Contextual factors 12 3
Contextual conditions 49 31
User attributes 4 2
Item features 7 12
Datasets
17
45. IntRS’15 - September 2015, Vienna, Austria
CoMoDa TripAdvisor
Domain Movies POIs
Rating scale 1-5 1-5
Ratings 2,098 4,147
Users 112 3,916
Items 1,189 569
Contextual factors 12 3
Contextual conditions 49 31
User attributes 4 2
Item features 7 12
Datasets
17
time,
daytype, season,
location, weather,
social, mood, …
46. IntRS’15 - September 2015, Vienna, Austria
CoMoDa TripAdvisor
Domain Movies POIs
Rating scale 1-5 1-5
Ratings 2,098 4,147
Users 112 3,916
Items 1,189 569
Contextual factors 12 3
Contextual conditions 49 31
User attributes 4 2
Item features 7 12
Datasets
17
age, gender, city,
country
47. IntRS’15 - September 2015, Vienna, Austria
CoMoDa TripAdvisor
Domain Movies POIs
Rating scale 1-5 1-5
Ratings 2,098 4,147
Users 112 3,916
Items 1,189 569
Contextual factors 12 3
Contextual conditions 49 31
User attributes 4 2
Item features 7 12
Datasets
17
director, country,
language, year, budget,
genres, actors
48. IntRS’15 - September 2015, Vienna, Austria
CoMoDa TripAdvisor
Domain Movies POIs
Rating scale 1-5 1-5
Ratings 2,098 4,147
Users 112 3,916
Items 1,189 569
Contextual factors 12 3
Contextual conditions 49 31
User attributes 4 2
Item features 7 12
Datasets
17
type, month and year
of the trip
49. IntRS’15 - September 2015, Vienna, Austria
CoMoDa TripAdvisor
Domain Movies POIs
Rating scale 1-5 1-5
Ratings 2,098 4,147
Users 112 3,916
Items 1,189 569
Contextual factors 12 3
Contextual conditions 49 31
User attributes 4 2
Item features 7 12
Datasets
17
user location, member
type
50. IntRS’15 - September 2015, Vienna, Austria
CoMoDa TripAdvisor
Domain Movies POIs
Rating scale 1-5 1-5
Ratings 2,098 4,147
Users 112 3,916
Items 1,189 569
Contextual factors 12 3
Contextual conditions 49 31
User attributes 4 2
Item features 7 12
Datasets
17
item type,
amenities, item
locality, price range,
hotel class, …
52. IntRS’15 - September 2015, Vienna, Austria
Evaluation Procedure
Overview
18
• Repeated random sub-sampling validation (20 times):
53. IntRS’15 - September 2015, Vienna, Austria
Evaluation Procedure
Overview
18
25% 50% 25%
Training set Candidate set Testing set
• Repeated random sub-sampling validation (20 times):
• Randomly partition the ratings into three subsets
54. IntRS’15 - September 2015, Vienna, Austria
Evaluation Procedure
Overview
18
25% 50% 25%
Training set Candidate set Testing set
• Repeated random sub-sampling validation (20 times):
• For each user-item pair (u,i) in the candidate set, compute the N most relevant
contextual factors and transfer the corresponding rating and context information ruic
in the candidate set to the training set as ruic' with c' ⊆ c containing the associated
contextual conditions for these factors
• Randomly partition the ratings into three subsets
55. IntRS’15 - September 2015, Vienna, Austria
Evaluation Procedure
Overview
18
25% 50% 25%
Training set Candidate set Testing set
• Repeated random sub-sampling validation (20 times):
• For each user-item pair (u,i) in the candidate set, compute the N most relevant
contextual factors and transfer the corresponding rating and context information ruic
in the candidate set to the training set as ruic' with c' ⊆ c containing the associated
contextual conditions for these factors
• Randomly partition the ratings into three subsets
56. IntRS’15 - September 2015, Vienna, Austria
Evaluation Procedure
Overview
18
25% 50% 25%
Training set Candidate set Testing set
• Repeated random sub-sampling validation (20 times):
• For each user-item pair (u,i) in the candidate set, compute the N most relevant
contextual factors and transfer the corresponding rating and context information ruic
in the candidate set to the training set as ruic' with c' ⊆ c containing the associated
contextual conditions for these factors
• Randomly partition the ratings into three subsets
57. IntRS’15 - September 2015, Vienna, Austria
Evaluation Procedure
Overview
18
25% 50% 25%
Training set Candidate set Testing set
• Repeated random sub-sampling validation (20 times):
• For each user-item pair (u,i) in the candidate set, compute the N most relevant
contextual factors and transfer the corresponding rating and context information ruic
in the candidate set to the training set as ruic' with c' ⊆ c containing the associated
contextual conditions for these factors
• Randomly partition the ratings into three subsets
58. IntRS’15 - September 2015, Vienna, Austria
Evaluation Procedure
Overview
18
25% 50% 25%
Training set Candidate set Testing set
• Repeated random sub-sampling validation (20 times):
• For each user-item pair (u,i) in the candidate set, compute the N most relevant
contextual factors and transfer the corresponding rating and context information ruic
in the candidate set to the training set as ruic' with c' ⊆ c containing the associated
contextual conditions for these factors
• Randomly partition the ratings into three subsets
59. IntRS’15 - September 2015, Vienna, Austria
Evaluation Procedure
Overview
18
25% 50% 25%
Training set Candidate set Testing set
• Repeated random sub-sampling validation (20 times):
• For each user-item pair (u,i) in the candidate set, compute the N most relevant
contextual factors and transfer the corresponding rating and context information ruic
in the candidate set to the training set as ruic' with c' ⊆ c containing the associated
contextual conditions for these factors
• Randomly partition the ratings into three subsets
60. IntRS’15 - September 2015, Vienna, Austria
Evaluation Procedure
Overview
18
25% 50% 25%
Training set Candidate set Testing set
• Repeated random sub-sampling validation (20 times):
• For each user-item pair (u,i) in the candidate set, compute the N most relevant
contextual factors and transfer the corresponding rating and context information ruic
in the candidate set to the training set as ruic' with c' ⊆ c containing the associated
contextual conditions for these factors
• Randomly partition the ratings into three subsets
• Measure user-averaged MAE (U-MAE), Precision@10 and Recall@10 on the testing
set, after training the prediction model on the new extended training set
61. IntRS’15 - September 2015, Vienna, Austria
Evaluation Procedure
Overview
18
25% 50% 25%
Training set Candidate set Testing set
• Repeated random sub-sampling validation (20 times):
• For each user-item pair (u,i) in the candidate set, compute the N most relevant
contextual factors and transfer the corresponding rating and context information ruic
in the candidate set to the training set as ruic' with c' ⊆ c containing the associated
contextual conditions for these factors
• Randomly partition the ratings into three subsets
• Measure user-averaged MAE (U-MAE), Precision@10 and Recall@10 on the testing
set, after training the prediction model on the new extended training set
• Repeat
62. IntRS’15 - September 2015, Vienna, Austria
user-item pair
top two contextual factors
rating transferred to training set
Evaluation Procedure
Example
19
+
+
=
rating in candidate set
63. IntRS’15 - September 2015, Vienna, Austria
(Alice, Skiing)
top two contextual factors
rating transferred to training set
Evaluation Procedure
Example
19
+
+
=
rating in candidate set
64. IntRS’15 - September 2015, Vienna, Austria
(Alice, Skiing)
Season and Weather
rating transferred to training set
Evaluation Procedure
Example
19
+
+
=
rating in candidate set
65. IntRS’15 - September 2015, Vienna, Austria
(Alice, Skiing)
Season and Weather
rating transferred to training set
Evaluation Procedure
Example
19
rAlice Skiing Winter, Sunny, Warm, Morning = 5+
+
=
66. IntRS’15 - September 2015, Vienna, Austria
(Alice, Skiing)
Season and Weather
Evaluation Procedure
Example
19
rAlice Skiing Winter, Sunny, Warm, Morning = 5
rAlice Skiing Winter, Sunny = 5
+
+
=
67. IntRS’15 - September 2015, Vienna, Austria
Baseline Methods for Evaluation
• Mutual Information (Baltrunas et al., 2012): given a user-item pair (u,i), it
computes the relevance score for the contextual factor Cj as the normalized
mutual information between the ratings for items belonging to i’s category
and Cj
• Freeman-Halton Test (Odić et al., 2013): calculates the relevance of a
contextual factor Cj using the Freeman-Halton test, which is the Fisher’s exact
test extended for contingency tables > 2 × 2
• Minimum Redundancy Maximum Relevance - mRMR (Peng et al., 2005):
ranks each contextual factor Cj according to its relevance to the rating
variable and redundancy to other contextual factors
• Random: randomly chooses the top N contextual factors for a user-item pair
20
68. IntRS’15 - September 2015, Vienna, Austria
Evaluation Results
U-MAE
21
CoMoDa
U-MAE
0.71
0.72
0.73
0.74
0.75
0.76
0.77
0.78
0.79
0.80
0.81
0.82
Number of Selected Contextual Factors
1 2 3 4
Largest Deviation Mutual Information Freeman-Halton mRMR Random All features
TripAdvisor
U-MAE
0.520
0.521
0.522
0.523
0.524
0.525
0.526
0.527
0.528
0.529
0.530
0.531
0.532
0.533
Number of Selected Contextual Factors
1 2 3
*
*
* * *
*
*
*
*
*
*
*
69. IntRS’15 - September 2015, Vienna, Austria
Evaluation Results
Precision@10
22
CoMoDa
Precision@10
0.0000
0.0002
0.0004
0.0006
0.0008
0.0010
0.0012
0.0014
0.0016
Number of Selected Contextual Factors
1 2 3 4
Largest Deviation Mutual Information Freeman-Halton mRMR Random All features
TripAdvisor
Precision@10
0.0100
0.0105
0.0110
0.0115
0.0120
0.0125
0.0130
0.0135
0.0140
0.0145
0.0150
0.0155
0.0160
Number of Selected Contextual Factors
1 2 3
*
*
*
*
*
* *
*
*
*
70. IntRS’15 - September 2015, Vienna, Austria
Evaluation Results
Recall@10
23
CoMoDa
Recall@10
0.000
0.002
0.004
0.006
0.008
0.010
0.012
0.014
0.016
Number of Selected Contextual Factors
1 2 3 4
Largest Deviation Mutual Information Freeman-Halton mRMR Random All features
TripAdvisor
Recall@10
0.100
0.105
0.110
0.115
0.120
0.125
0.130
0.135
0.140
0.145
0.150
0.155
0.160
Number of Selected Contextual Factors
1 2 3
*
*
*
*
* *
*
*
*
*
71. IntRS’15 - September 2015, Vienna, Austria
Evaluation Results
Practical Implications
• Using Largest Deviation, we know that we can ask only the contextual factors
C1, C2 and C3 when we ask user u to rate item i
24
72. IntRS’15 - September 2015, Vienna, Austria
Outline
25
• Introduction
• Related Works
• Selective Context Acquisition
• Experimental Evaluation and Results
• Conclusions and Future Work
73. IntRS’15 - September 2015, Vienna, Austria
Conclusions
• Identifying which contextual factors should be acquired from the user upon
rating an item is an important and practical problem for CARSs
• We tackled this problem with a new method that asks the user to specify
those contextual factors that if considered in the CARS prediction model
would produce a rating prediction that is most different from the context-free
prediction
• Results from our offline experiment confirm that the proposed parsimonious
context acquisition strategy elicits ratings with contextual information that
improve more the recommendation performance
26
74. IntRS’15 - September 2015, Vienna, Austria
Future Work
• Evaluate the performance of employing an Active Learning method for
adaptively selecting both the item to rate and the contextual information to
add
• Understand how the proposed method can be extended to generate requests
for contextual data that takes into account possible correlations between
contextual factors
• Update the evaluation procedure so that it can be used also on rating
datasets for which only a subset of contextual factors is known
• Integrate the developed method into our STS app and perform a live user
study
27