Can Trailers Help to Alleviate Popularity Bias in Choice-Based Preference Elicitation?

Can Trailers Help to
Alleviate Popularity Bias in
Choice-Based Preference
Elicitation?
Mark P. Graus
Martijn C. Willemsen
Human-Technology
Interaction Group
Eindhoven University
of Technology

Summary
• We wanted to see if we could make people chose
less popular items in a choice-based preference
elicitation recommender system by showing them
trailers.
• We tested this in a between subjects user study.
• We found that after watching trailers people chose
less popular items, while user experience was not
negatively affected.
9/16/2016
Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @
RecSys '16
2

Motivation
9/16/2016
RecSys '16
3

Latent Feature Diversification
• Can we reduce choice overload
through diversification based on
the latent features of a matrix
factorization model?
Willemsen, M. C., Graus, M. P., & Knijnenburg, B.
P. (2016). Understanding the role of latent feature
diversification on choice difficulty and satisfaction.
User Modeling and User-Adapted Interaction, 1–
43. http://doi.org/10.1007/s11257-016-9178-6
Latent Feature 1
LatentFeature2
9/16/2016
RecSys '16
4

Latent Feature Diversification Findings
9/16/2016
RecSys '16
5
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
low mid high
standardizedscore
diversification
Perceived diversity
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
low mid high
standardizedscore
diversification
Expected choice difficulty

LatentFeature2
Latent Feature 1
4) Iteration 2
Choice-Based Preference Elicitation
• Can we improve the user experience
during cold start by having people
choose between items instead of
rating items?
Graus, M. P., & Willemsen, M. C. (2015). Improving the
User Experience during Cold Start through Choice-
Based Preference Elicitation. In Proceedings of the 9th
ACM Conference on Recommender Systems - RecSys
’15 (pp. 273–276). New York, New York, USA: ACM
Press. http://doi.org/10.1145/2792838.2799681
9/16/2016
RecSys '16
6

How does this work? Step 1
Latent Feature 1
LatentFeature2
Iteration 1a: Diversified choice set is
calculated from a matrix factorization
model (red items)
Iteration 1b: User vector (blue arrow) is moved
towards chosen item (green item), items with
lowest predicted rating are discarded (greyed
out items)
9/16/2016
RecSys '16
7

How does this work? Step 2
Iteration 2: New diversified choice set
(blue items)
End of Iteration 2: with updated vector and
more items discarded based on second choice
(green item)
9/16/2016
RecSys '16
8

Choice-Based Preference Elicitation
Findings
• People are more satisfied with choice-based than
rating-based interfaces
• This comes mainly because of increased popularity
(items with many ratings)
But we do not want to
recommend popular
items!
9/16/2016
RecSys '16
9
Satisfaction
with Chosen
Item
Popularity
Difficulty
Intra List
Similarity
-2.407(.381)
p<.001
-.240 (.145)
p<.1
-.479 (.111)
p<.001
-.257 (.045)
p<.001
14.00 (4.51)
p<.01
Choice-
Based List
+
+
- -
+

Why do people end up with popular
items?
• Our hypothesis
• Users don’t know all movies, hard to judge based on
metadata alone
• People choose movies they know
• People know movies that are popular
• Choosing popular movies results in popular recommendations
• Our Solution
• Provide trailers as additional information for making choices
9/16/2016
RecSys '16
10

Rationale
• In the music domain
• Implicit Feedback
• Movie domain
• Implicit feedback is sparse
• I (can) listen to 100s of tracks in a week, but I can’t
watch 100s of movies a week (and sustain my job).
• We can approximate experiencing movies by
providing trailers
9/16/2016
RecSys '16
11

Study
9/16/2016
RecSys '16
12

Choice-Based Interface with or without
Trailers
• N = 71
9/16/2016
RecSys '16
13

Expected Effects
Trailers
Perceived
Diversity
Informativeness
Perceived
Novelty
Choice
Satisfaction
System
Satisfaction
Popularity of
Chosen Items
- +
+
+?
-?-
-
+
9/16/2016
RecSys '16
14

Set Up
• Random assignment
• Choice Based Preference Elicitation – 9 choices of 10 items
[with/without trailers]
• Recommendation List – Top-10 Items [with/without trailers]
• Survey to measure User Experience
• Informativeness
• Perceived Diversity
• Perceived Novelty
• System Satisfaction
• Choice Satisfaction
9/16/2016
RecSys '16
15

Results
9/16/2016
RecSys '16
16

Do trailers affect the popularity of chosen
items?
• Checked through repeated measures (10 choices)
• Popularity is expressed as the rank ordering by
number of ratings in MovieLens dataset
• Trailers do not decrease popularity of choices
• The popularity rank of the item chosen in each choice
set
• The average popularity rank of all items in each choice
set
9/16/2016
RecSys '16
17

However: Relative Popularity of Choice
• average popularity rank of choice set – popularity
rank of chosen item
9/16/2016
RecSys '16
18

If we look at people that actually watched
trailers
• People that watch trailers are
more likely to pick less popular
movies from the lists
9/16/2016
RecSys '16
19

User Experience
Choice
Satisfaction
Perceived
Diversity
System
Satisfaction
Informativeness
.570 (.295)
p < 0.1
-.604 (.091)
p < 0.01
.244 (.162)
n.s.
.785 (.115)
p < 0.01
-.266 (.122)
p < 0.05
Trailers
.611 (.256)
p < 0.05
-.570 (.259)
p < 0.05
-
+
-
-
+
9/16/2016
+
RecSys '16
20

Conclusions
9/16/2016
RecSys '16
21

What we found
• Providing people with trailers does make them
choose less popular items.
• No indication that the overall satisfaction is affected
negatively or positively
• As opposed to initial study where popularity resulted in
increased satisfaction
9/16/2016
RecSys '16
22

Limitations
• When were trailers watched?
• In the preference elicitation task?
• In the decision task?
Future Work
• How do trailers affect a more standard rating
interface?
9/16/2016
RecSys '16
23

Thank You
• Questions/remarks?
Mark Graus – PhD Student
Human-Technology Interaction Group
Eindhoven University of Technology
m.p.graus@tue.nl
https://twitter.com/newmarrk
https://linkedin.com/in/markgraus
http://www.marrk.nl
9/16/2016
RecSys '16
24

Can Trailers Help to Alleviate Popularity Bias in Choice-Based Preference Elicitation?

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (12)

Dernier

Dernier (20)

Can Trailers Help to Alleviate Popularity Bias in Choice-Based Preference Elicitation?