'Meaning is its use' - Towards the use of distributional semantics for content-based recommender systems

SASWeb 2012 - Workshop on Social, Semantic and Adaptive Web
Montréal (Canada), 16.07.2012

‘Meaning is its use’: towards the use
of distributional semantics for
content-based recommender systems
Cataldo Musto, Ph.D.
University of Bari Aldo Moro (Italy) - cataldo.musto@uniba.it

“meaning
is its use”

L.Wittgenstein
(Austrian philosopher)
Cataldo Musto, ‘Meaning is its use’: towards the use of distributional semantics for content-based recommender systems. SASWeb Workshop, UMAP 2012, 16.07.12

semantics:
study of meaning

Greek: σημαντικος

semantics plays a key role for
most of the adaptive systems.


adaptive systems can beneﬁt
from semantic representation
of the information.


example.


Recommender Systems

Relevant items (movies, news, books, etc.) are pushed to the
user according to her preferences or her needs.

content-based recommenders
Suggest items similar to those liked in the past by the user


scenario.
book recommendation


content-based recommenders
book recommendation: key concepts

• Each book has to be described through a set of
textual features
• e.g title of the book, summary, etc.
• Each user is described through textual features, as
well

• Recommendations are provided by calculating
the overlap between the textual description of the
TV show and the features stored in the user proﬁle

“I love turkey.
I will choose it for my holidays”

?

“what book will the
user be interested in?”

vs.


how can we boost
content-based recommender systems
with semantics ?


alternative representation

distributional models
(Firth, 1957)

Firth, J.R. A synopsis of linguistic theory
1930-1955. In Studies in Linguistic Analysis,
pp. 1-32, 1957.


insight
by analyzing large corpus of textual data it is possible
to infer information about the usage (about the meaning)
of the terms.


insight
of the terms.
example


insight
of the terms.

distributional hypothesis
“ words that share similar contexts
(usages) share similar meaning “


• Key: deﬁnition of what is the
‘context’
• Different granularities
are possible
• Document
• Paragraph
• Sentence
• Sliding window of words

term/context matrix (WordSpace)

c1 c2 c3 c4 c5 c6 c7 c8 c9

t1 ✔ ✔ ✔ ✔

t2 ✔ ✔ ✔ ✔

t3 ✔ ✔ ✔

t4 ✔ ✔ ✔ ✔


WordSpace
example

beer
wine

glass

spoon


beer vs. glass: good overlap

c1 c2 c3 c4 c5 c6 c7 c8 c9

t1 ✔ ✔ ✔ ✔
t2 ✔ ✔ ✔ ✔

t3 ✔ ✔ ✔
t4 ✔ ✔ ✔ ✔


beer vs. spoon: no overlap

c1 c2 c3 c4 c5 c6 c7 c8 c9

t1 ✔ ✔ ✔ ✔

t2 ✔ ✔ ✔ ✔

t3 ✔ ✔ ✔

t4 ✔ ✔ ✔ ✔


representation of documents can be inferred
by combining the representation of the terms
occurring in the document.


term/context matrix (DocSpace)

c1 c2 c3 c4 c5 c6 c7 c8 c9

t2 ✔ ✔ ✔ ✔

t3 ✔ ✔ ✔

d1 ✔ ✔ ✔ ✔ ✔


similarity between documents (DocSpace)

c1 c2 c3 c4 c5 c6 c7 c8 c9

d1 ✔ ✔ ✔ ✔ ✔

d2 ✔ ✔ ✔

d3 ✔ ✔

d4 ✔ ✔ ✔ ✔ ✔


similarity between documents: good overlap

c1 c2 c3 c4 c5 c6 c7 c8 c9

d1 ✔ ✔ ✔ ✔ ✔

d2 ✔ ✔ ✔

d3 ✔ ✔

d4 ✔ ✔ ✔ ✔ ✔


similarity between documents: no overlap

c1 c2 c3 c4 c5 c6 c7 c8 c9

d1 ✔ ✔ ✔ ✔ ✔

d2 ✔ ✔ ✔

d3 ✔ ✔

d4 ✔ ✔ ✔ ✔ ✔


recap
models for representing terms/
documents in large vector spaces

light semantics
it is simple to calculate
similarities between words
and document


strength:
representation based on
distributional models are
inherently multilingual.


multilingual representation

• Assumption
• The distribution of the terms is (almost) language-
independent

drink bere

beer / birra
glass bicchiere



• Assumption
• The distribution of the terms is (almost) language-
independent

The position of concept of in a WordSpace will be beer
always the same, regardless the language!


(english) WordSpace

beer
wine

spoon
dog


(italian) WordSpace
relationships between
terms stay
birra regardless the
language!
vino

cucchiaio
cane


comes with no costs.
Thanks to distributional hypothesis.


recap
models for representing terms/
documents in large vector spaces

light semantics
it is simple to calculate
similarities between words
and document

representation is inherently
multilanguage

how to combine
with
content-based recommender systems?


a novel recommendation framework based on VSM

eVSM
enhanced Vector Space Model


eVSM building blocks

distributional models.

eVSM representation
mystery book

poetry book

recipe book
recipe book


user proﬁle
how to represent it?

• In eVSM each item is represented as a vector
• User proﬁle vector space representation as well needs a

• How?

• For example, by combining vectors of the items (documents)
the user liked in the past


user proﬁle

Items Rating Threshold

VSM representation of RI-based proﬁle for user u

eVSM scenario
mystery book
user proﬁle

poetry book

recipe book
recipe book


eVSM scenario
Recommendation
task seen as
mystery book
similarity
user proﬁle calculation
between vectors
poetry book

recipe book
recipe book


eVSM scenario
recommender
mystery book
system suggests
user proﬁle mystery book.

poetry book

recipe book
recipe book


Why a multilanguage
representation does matter?


Language issues

• VSM representation is language-dependant
• User proﬁle built in a language can not be
exploited to provide recommendation of
items described in another language


eVSM
language-dependant recommendations

user proﬁle content-based recommendations


Vector Space Model
multilingual scenario

d1 d2 d3 p

t1 basketball italian ✔
✔
document
t2 italian ✔ ✔

t3 bargnani ✔ ✔

t4 pallacanestro ✔ ✔

t5 italiana ✔ ✔


Vector Space Model

d1 d2 d3 p

t1 basketball english ✔ ✔

t2 italian documents ✔ ✔

t3 bargnani ✔ ✔


t5 italiana ✔ ✔


Vector Space Model

d1 d2 d3 p

t1 basketball ✔ ✔

t2 italian ✔ ✔

t3 bargnani ✔ ✔


t5 italiana ✔ ✔


eVSM

d1 d2 d3 p

t1 basketball user interested in
✔ ✔

t2 italian
basketball, italian
✔ ✔
language
t3 bargnani ✔ ✔


t5 italiana ✔ ✔


eVSM

d1 d2 d3 p


t2 italian ✔ ✔

t3 bargnani ✔ ✔

t4 pallacanestro
✔ ✔
t5 italiana
✔ ✔

eVSM

d1 d2 d3 p

X
t2 italian ✔ ✔
X
t3 bargnani ✔ ✔


t5 italiana ✔ ✔


eVSM overcomes this issue.


eVSM
providing suggestions - multilingual scenario
DocSpace for L1 DocSpace for L2
c1 c2 c3 c4 c5 . . . ck c1 c2 c3 c4 c5 . . . ck

d1 Parallel d1
DocSpaces
d2 d2
Built upon the
d3 same d3

d4
set of content d4

d5 d5

(italian) (english)

eVSM

italian football news
english football news
italian user proﬁle
english basketball news
italian basketball news
english politics news
italian politics news
english politics
news

(italian) (english)


eVSM
c1 c2 c3 c4 c5 . . . ck c1 c2 c3 c4 c5 . . . ck

d1 Parallel d1
DocSpaces
d2 d2
Built upon the
d3 same d3

d4
set of content d4

p
d5
L1

user proﬁle in L1
(italian)

eVSM
c1 c2 c3 c4 c5 . . . ck c1 c2 c3 c4 c5 . . . ck

d1 Parallel d1
DocSpaces
d2 d2
Built upon the
d3 same d3

d4
set of content d4

p p
L1 L1

we can project user proﬁle in the
DocSpace of english items

eVSM

italian football news
english football news
italian user proﬁle italian user proﬁle
english basketball news
italian basketball news
english politics news
english politics
news

(italian) (english)
through similarity calculations an english news
about basketball is received as recommendation!

eVSM
multilanguage recommendations

italian

english


recap and contributions.

adaptive systems.

semantics.

recommender systems.

eVSM

richer representation based on

framework for multilingual recommendations


experimental evaluation

applications


applications

‘in vitro’
experiments

Cataldo Musto - Enhanced Vector Space Models for Content-based Recommender Systems - Ph.D. defense - University of Bari Aldo Moro, Italy - 08.06.12

movie recommendation
‘in vitro’ experiments
• Goal: to provide users with recommendations about movies
worth to be watched.
• Subset of 100k MovieLens dataset + Wikipedia content
• Monolingual and Multilingual settings


experimental design
experiments

• Experiment
• How do the model perform with respect to other
state of the art approaches?
• VSM - Vector Space Model
• LSI - Latent Semantic Indexing
• Bayes Bayes Text Classiﬁer


experiment
87
Movielens dataset

eVSM VSM
86.25
85.94 86.01 LSI Bayes
85.58 85.52
85.5 85.39
85.27

84.97
84.85
84.77 84.75
84.75 84.7 84.7
84.58
84.47 84.5
84.43

84
p@1 P@3 P@5 P@10

Gap always around 1%

experiment
87
Movielens dataset

eVSM VSM
86.25
85.94 86.01 LSI Bayes
85.58 85.52
85.5 85.39
85.27

84.97
84.85
84.77 84.75
84.75 84.7 84.7
84.58
84.47 84.5
84.43

84
p@1 P@3 P@5 P@10

Signiﬁcant Improvement

applications

‘in vivo’
experiments


Play.me
personalized music playlists
• Goal

• To provide users with personalized music playlists

• Methodology

• Extraction of explicit user preferences from Facebook

• Playlist creation by enriching explicit user preferences with similar artists.

• Comparison of two enrichment algorithms

• DBPedia-based enrichment

• Distributional models-based enrichment


Play.me
architecture


Extractor
insight

Social Media provide us unlimited, trustful
and continously updated ﬂow of information about
user interests and needs.

Extractor
insight

Social Media are a cheap and effective way
to overcome cold start.

Myusic
data extraction from Facebook


Myusic

explicit preferences

Myusic

implicit preferences

Play.fm
enrichment

• Given a set of explicit preferences extracted from
Facebook
• Play.me enrichs this set by calculating artists similar
to those the user explicity likes


Play.fm
enrichment example

Coldplay extracted from Facebook
enrichment

radiohead red hot chili peppers kings of leon


Play.fm
enrichment

• Comparison of two approaches
•
Linked Data
•
Distributional Models


Linked Open Data Cloud

Structured
(RDF)
representation
of the information
stored in Wikipedia.

Play.fm
enrichment based on Distributional Models

• Distributional Models
• Each artist is represented through a set of
tags
• Each artist is represented as a point in a
distributional DocSpace
• Similarity calculations to extract the most
similar artists.


Play.fm

Coldplay
Radiohead
Kings of Leon

Lady Gaga


Play.fm

input: vector space representation

output: artists with the highest cosine similarity

radiohead the killers kings of leon


Play.me
playlist

Most popular songs of the artists extracted from Facebook (as well
as those added through the enrichment) are proposed to the user.


experimental design

• Experiment
• Which one is the enrichment technique that
can provide users with the best playlists ?
• User study with 30 users.


experimental design
results
76.3
80

75.2 Linked Data
Distributional Models
Baseline (Popularity)
73.75
69.7
67.5

65.9
64.6
61.25 63.2

58 58 58
55
m=1 m=2 m=3

m = number of artists added for each extracted artist

conclusions.


eVSM overcomes
state of the art approaches

semantic representation based
on distributional models
effectively tackles language-issues of CBRS

end.

questions?


'Meaning is its use' - Towards the use of distributional semantics for content-based recommender systems

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (20)

Similaire à 'Meaning is its use' - Towards the use of distributional semantics for content-based recommender systems

Similaire à 'Meaning is its use' - Towards the use of distributional semantics for content-based recommender systems (20)

Plus de Cataldo Musto

Plus de Cataldo Musto (20)

Dernier

Dernier (20)

'Meaning is its use' - Towards the use of distributional semantics for content-based recommender systems

Notes de l'éditeur