Lecture 2: Computational Semantics

Seman&c
Analysis
in
Language
Technology

http://stp.lingﬁl.uu.se/~santinim/sais/2014/sais_2014.htm 
 
Computa(onal
Seman(cs

Marina
San(ni

san$nim@stp.lingﬁl.uu.se

Department
of
Linguis(cs
and
Philology

Uppsala
University,
Uppsala,
Sweden

Autumn
2014

Lecture 2: Computational Semantics
1

Outline

•  Formal
Representa(ons
and
Computa(onal

approaches

–  The
Seman(cs
of
First-‐Order
Logic

–  Event
Representa(ons

–  Descrip(on
Logics
&
the
Web
Ontology
Language

–  Syntax-‐Driven
Seman(c
Analysis:
Composi(onality

•  Corpus-‐based
approaches

–  Latent
Seman&c
Analysis

–  Topic
models

–  Distribu&onal
Seman&cs…

2

Generally
speaking,
seman(cs
and
meaning…

In
linguis(cs…

•  Seman&cs
is
the
study
of
meaning

•  Meaning
is
the
core
of
human
communica(on.
It
is

the
msg
that
we
want
to
convey
(explicity
or

implicitly)

•  Meaning
representa&ons
are
formal
structures

•  Meaning
representa&on
languages
are
frameworks

that
speﬁcy
the
syntax
and
seman(cs
of
these

representa(ons

3

(Computa(onal)
Seman(cs
vs

Pragma(cs

•  Roughly,
seman(cs
is
the
meaning
that
can
be

deduced
directly
from
an
expression,
with
no

extra-‐linguis(c
informa(on.

– cf:
”the
sun
is
rising”
vs
”the
bus”

•  Computa(onal
Seman(cs
focuses
not
only
on

the
abstract
accounts
of
meanings,
but
also
in

a
concrete
formaliza(ons
that
can
support

implementa&on

4

Seman(c
Analysis…

…
is
the
process
that
we
use
to

– create
representa(ons
of
meaning

– assign
them
to
linguis(c
inputs

5

WHAT
IS
NEEDED
IN
A
MEANING

REPRESENTATION?

Ch
17

6

The
Representa(on
of
Meaning

•  Meaning
of
linguis(c
expressions
can
be
captured
in

formal
structures
that
we
call
meaning

representa&ons.

•  What
we
need
are
representa&on
that
bridge
the
gap

from
linguis&c
inputs
to
the
non
linguis&c
knowledge
of

the
world

•  It
requires
access
to
the
representa&ons
that
link
the

linguis&c
elements
involved
in
the
task
to
the
non-‐
linguisitc
’knowledge
of
the
world’
needed
to
perform

the
task.

7

Seman(c
processing…

”Learning
to
use
a
new
piece
of
soWware
by

reading
a
manual”

– knowledge
about
current
computers

– similar
soWware
applica(ons

– knowledge
about
users
in
general

8

Requirements

•  The
basic
requirements
that
a
meaning

respresenta(on
must
fulﬁll:

– Veriﬁability

– Ambiguity

– Inference

– Expressiveness

9

First-‐Order
Logic

•  FOL
is
a
computa(onally
tractable
approach
to

the
representa(on
of
knowledge
that
sa(sﬁes

many
of
the
previous
requirements,
namely:

– Veriﬁability

– Inference

– Expressiveness

10

FOL
(Wikipedia)

http://en.wikipedia.org/wiki/First-order_logic

•  First-‐order
logic
is
a
formal
system
used
in

mathema(cs,
philosophy,
linguis(cs,
and

computer
science.

•  It
is
also
known
as:

– 
ﬁrst-‐order
predicate
calculus

– the
lower
predicate
calculus

– quan&ﬁca&on
theory

– predicate
logic

– etc.

11

Why
”ﬁrst-‐order”?

12
There are more
powerful forms of
logic, but ﬁrst-‐‑
order logic is
adequate for most
everyday
reasoning.

FOL

•  First-‐order
logic
is
symbolized
reasoning
in

which
each
sentence,
or
statement,
is
broken

down
into
a
subject
and
a
predicate.

•  The
predicate
modifies
or
defines
the

proper(es
of
the
subject.

•  In
first-‐order
logic,
a
predicate
can
only
refer

to
a
single
subject.

13

But…
undecidable
(some(mes)

•  The
Incompleteness
Theorem
,
proven
in

1930,
demonstrates
that
ﬁrst-‐order
logic
is
in

general
undecidable.

•  That
means
there
exist
statements
in
this
logic

form
that,
under
certain
condi(ons,
cannot
be

proven
either
true
or
false.

•  Ex:
can’t
solve
the
Hal(ng
Problem

14

Hal(ng
Problem

•  In
1936
Alan
Turing
proved
that
it's
not
possible
to
decide
whether

an
arbitrary
program
will
eventually
halt,
or
run
forever.

•  The
official
defini(on
of
the
problem
is
to
write
a
program
(actually,

a
Turing
Machine*)
that
accepts
as
parameters
a
program
and
its

parameters.
That
program
needs
to
decide,
in
finite
(me,
whether

that
program
will
ever
halt
running
these
parameters.

•  The
hal(ng
problem
is
a
cornerstone
problem
in
computer
science.

It
is
used
mainly
as
a
way
to
prove
a
given
task
is
impossible,
by

showing
that
solving
that
task
will
allow
one
to
solve
the
hal(ng

problem.

*A
Turing
machine
is
a
hypothe(cal
device
that
manipulates
symbols

according
to
a
table
of
rules.
Despite
its
simplicity,
a
Turing
machine

can
be
adapted
to
simulate
the
logic
of
any
computer
algorithm,

15

Representa(on

•  A
sentence
in
ﬁrst-‐order
logic
is
wrifen
in
the

form
Px
or
P(x),
where
P
is
the
predicate
and
x

is
the
subject,
represented
as
a
variable.

•  Complete
sentences
are
logically
combined

and
manipulated
according
to
the
same
rules

as
those
used
in
Boolean
algebra.

16

FOL’s
machinery

•  Terms:

– Constants

– Func(ons

– Variables

•  Logical
connec(ves

•  Quan(ﬁers

•  Lambda
nota(on

17

The
Seman(cs
of
FOL

•  Truth
table

•  Inference

18

Predicates
and
terms

•  John
is
a
sailor

sailor(j)

•  In
FOL
we
can
represent
the
informa(on

conveyed
by
NL
entences
sta(ng
that
an
object
is

a
member
of
a
certain
set
by
means
of
a

predicate
such
as
”sailor”
(deno(ng
a
set
of

object),
and
a
term
such
as
J,
deno(ng
John.

•  The
atomic
formula
sailor(j)
expresses
the

statement.

19

Arity

•  Using
predicates
of
higher
arity,
we
can
also

assign
a
seman(c
interpreta(on
to
sentences

sta(ng
that
certain
objects
stand
in
certain

rela(on:

•  John
likes
Mary

like(j,m)

20

Universal
quan(ﬁer:
∀

•  The
seman(c
interpreta(on
of
sentences
asser(ng

that
a
set
is
included
in
another
can
be
expressed

by
means
of
a
universal
quan(ﬁer
∀

Dogs
are
mammals

∀xdogxàmammals(x)!
21

Existen(al
quan(ﬁer:
Ǝ

•  The
existen(al
quan(ﬁer
Ǝ
can
be
used
to

capture
the
informa(on
that
a
certain
set
is

not
empty,
as
epressed
by
the
sentence:

I
have
a
car

Ǝxcar(x)∧own(spkr,x)!
22

3
Connec(ves:
∧∨¬

John
and
Mary
are
happy

happy(j)
∧
happy(m)

John
is
not
married

¬married(j)

In
certain
applica(ons,
represen(ng
this
info
is
all
we

need
(eg.
enquiry
system
for
train
transporta(on:
a

person
travelling
from
sta(on
a)
to
sta(on
b)

23

λ

nota(on
&
λ
reduc(on

•  It
is
a
way
to
”abstract”
from
FOL
formulae

•  λ
followed
by
one
or
more
variables,
followed

by
a
FOL
formula
that
makes
use
of
these

variables.

•  Basically:
manipula(on
and
aggrega(on
of

variables.

24

Example:
lambda
expressions

•  λx.λy.Near(x,y)
=
something
near
something
else

•  λx.λy.Near(x,y)(uppsala)

–  Reduc(on:
λy.Near(uppsala,y)

•  λy.Near(uppsala,y)
(stockholm)

–  Reduc(on:
Near(uppsala,stockholm)

•  More:
Sec(ons
17.3.3
and
18.3;
see
also
hfps://ﬁles.nyu.edu/cb125/public/Lambda/

25

Proof
Theory

•  What
makes
FOL
a
logic
is
that
it
also
includes

a
speciﬁca(on
of
the
valid
conclusions
that

can
be
derived
from
the
info.

a)  All
trains
depar(ng
from
Stockholm
and

arriving
at
Gävle
stop
at
Uppsala

b)  Train
531
departs
from
S
and
arrives
at
G.

c)  Train
531
stops
at
U

26

Inference
rules

1.  ∀x(train(x)∧depart(x,S)arrive(x, G) à stop(x, U)!
2.  train(t531)∧depart(t531),S)∧arrive(t531,G)!
3.  stop(t531,U)!
•  An
inference
rule
consists
of
a
set
of
statements

called
premises
and
a
statement
called
conclusion.

The
inference
rule
is
a
claim
that
if
all
premises
are

true,
then
the
conclusion
is
true.

27

Ex:
Modus
ponens
=
if-‐then
reasoning

•  It
is
an
example
of
a
valid
inference
rule:

– If
P
is
the
case,
and
P
à
Q
is
the
case,
than
Q
is

the
case.

28

Cf.
Proposi(onal
logic
(wikipedia)

http://en.wikipedia.org/wiki/Aristotelian_logic

•  Syllogism
and
inference:

–  Men
are
mortal
=
A

–  Socrates
is
a
man
=
B

–  Socrates
is
mortal
=
C

Proposi(onal
logic
(also
called
senten(al
logic)
is
the
logic
the
includes
sentence
lefers

(A,B,C)
and
logical
connec(ves,
but
not
quan$fiers.

The
seman(cs
of
proposi(onal
logic
uses
truth
assignments
to
the
lefers
to
determine

whether
a
compound
proposi(onal
sentence
is
true.

The
syllogism
is
an
inference
in
which
one
proposi(on
(the
"conclusion")
follows
of

necessity
from
two
others
(the
"premises").
A
proposi(on
may
be
universal
or
par(cular,

and
it
may
be
affirma(ve
or
nega(ve.

Syntac(cally,
first-‐order
logic
has
the
same
connec(ves
as
proposi(onal
logic,
but
it
also

has
variables
for
individual
objects,
quan(fiers,
symbols
for
func(ons,
and
symbols
for

rela(ons.
The
seman(cs
include
a
domain
of
discourse
for
the
variables
and
quan(fiers
to

range
over,
along
with
interpreta(ons
of
the
rela(on
and
func(on
symbols.

29

Many
Logic-‐s

•  logic
of
sentences
(proposi(onal
logic),

•  logic
of
objects
(predicate
logic),

•  logic
involving
uncertain(es,

•  logic
dealing
with
fuzziness,

•  temporal
logic
etc.

30

Prac(cal
use
Of
Modus
Ponens

•  Forward
chaining

–  Top-‐down:
As
soon
as
a
new
fact
is
added
to
the

knowledge
base,
all
applicable
rules
are
found
and

applied,
each
esul(ng
n
the
addi(on
of
new
facts
to

then
KB.
Drawback:
facts
that
will
never
be
needed

are
deduced
and
stored

•  Backward
chaining:

–  Bofom
up:
run
in
reverse
to
prove
speciﬁc

proposi(ons
are
true
(à
PROLOG).

•  Both
incomplete:

–  Ie,
there
valid
inferences
that
cannot
be
found
by

systems
that
use
these
methods
alone.

31

State
and
Event
Representa(ons

•  States
and
events

– States
are
condi(ons,
or
proper(es,
that
remain

unchanged
over
a
period
of
(me

– Events
denote
changes
in
some
state
of
aﬀairs

32

Predicates

•  Predicates
in
FOL
have
fixed
arity:
they
take
a
fixed

number
of
arguments
–
predicates
have
a
fixed

arity

33

Possible
solu(on

•  event
variables
à
(neo)
Davidsonian
event

representa(on

Ǝe eating(e) ∧ eater(e, speaker)∧ eaten(e,turkey sandwich) ∧
meal(e,lunch) ∧ location(e,desk)∧time(e,tuesday)#
•  No
need
to
specify
a
ﬁxed
number
of
arguments

•  The
event
itself
is
a
single
argument.

•  Everything
else
is
captured
by
addi(onal
predica(on

34

Descrip(on
Logics

•  DLs
refer
to
a
family
of
logical
approaches
that
corrispond
to

diﬀerent
subsets
of
FOL.

•  We
can
use
DLs
to
model
an
applica(on
domain.
The
focus
is
then

on:

–  Representa(on
of
knowledge
about
categories

–  The
set
of
categories
in
an
applica(on
domain
is
called
terminology

–  The
terminology
is
arranged
in
a
hierachical
organiza(on
called

ontology,
which
capture
superset
&
subset
rela(ons
among
categoires/
concepts.

–  In
order
to
specify
a
hierachical
structure,
we
can
use
subsump$on

rela(ons
betw
the
appropriate
concepts
in
a
terminiology

–  Subsump$on
is
a
form
of
inference.
Determines
whether
a
suprset/
subset
rela(on
(based
on
the
fact
asserted
in
a
terminology)
exists
betw

two
concepts.

35

OWL
and
the
Seman(c
Web

•  A
Descrip(on
Logic
roughly
similar
to
the
previous

example
is
used
in
the
Web
Ontology
Language
(OWL).

•  OWL
is
a
language
used
for
the
develoment
of

ontologies
that
should
encapsulate
the
knowledge
in

the
development
of
the
Seman(c
Web

•  The
Seman(c
Web
is
the
eﬀort
to
formally
specify
the

seman(cs
of
the
contents
of
the
web
.

à
lect
9

36

Seman(c
web
(wikipedia)

hfp://en.wikipedia.org/wiki/Seman(c_Web

•  The
Seman(c
Web
is
a
collabora(ve
movement
led
by

interna(onal
standards
body
the
World
Wide
Web

Consor(um
(W3C).

•  By
encouraging
the
inclusion
of
seman(c
content
in
web

pages,
the
Seman(c
Web
aims
at
conver(ng
the
current

web,
dominated
by
unstructured
and
semi-‐structured

documents
into
a
"web
of
data".

•  Web
3.0

–  Tim
Berners-‐Lee
has
described
the
seman(c
web
as
a

component
of
"Web
3.0".

–  "Seman(c
Web"
is
some(mes
used
as
a
synonym
for
"Web

3.0",
though
each
term's
deﬁni(on
varies.

37

TECHNIQUES
FOR
ASSIGNING

MEANINGS
TO
LINGUISTIC
INPUT

J&M
-‐
Ch
18

see
also
Saeed,
Ch
10:
Formal
se

38

Syntax-‐Driven
Seman(c
Analysis

•  :
Meaning
representa(ons
are
assigned
to

sentences
on
the
basis
of
knowledge
taken

from
the
lexicon
and
grammar

39

Principle
of
Composi(onality

•  PoC:
the
meaning
of
a
sentence
can
be

constructed
from
the
meaning
of
its
parts.

•  Watch
out!
the
meaning
of
a
sentence
is
not

based
only
on
the
words
that
make
it
up,
but
also

on
the
ordering
and
grouping
of
words
and
on

the
rela(ons
among
the
words
in
the
sentence.

•  Basically,
the
meaning
of
a
sentence
is
par(ally

based
on
its
syntac(c
structure.

40

The
rule-‐to-‐rule
hypothesis

•  we
do
not
define
languages
by
enumera(ng

the
meanings
that
are
permifed.

•  But
we
define
a
finite
set
of
devices
that

generate
the
correct
meaning
for
the
context.

•  These
devices
are
based
on
grammar
rules

and
lexical
entries.

41

Two
constrained
approaches

1.  The
ﬁrst
is
based
on
FOL
and
lambda-‐
nota(on.

2.  The
second
is
based
on
feature-‐structure
and

uniﬁca(on

42

1:
FOL

•  Every
restaurant
has
a
menu,
2
meanings:

– All
restaurants
have
a
menu

– There
is
a
menu
in
the
world
and
all
the
restarrants

share
it

43

1.
Quan(ﬁer
scope
ambiguity

•  Expressions
containing
quan(ﬁers
can
create

ambiguity
even
if
there
is
no
syntac(c,
lexical

or
analphoric
ambiguity.

44

Underspecifica(on
and
storage

•  The
restaurant
fills
the
haver
role
and
the
menu
fills
the

had
role.

•  it
remain
agnos(c
about
the
placement
of
the

quan(fies

45
We
use
λ-‐expressions

and
a
store.

The
quan(fied
expressions
are
in

the
form
of
λ-‐‑expressions thant

can
be
combined
with
the
core

representaton
in
the
right
way.

We
have
access
to
the
quan(fier
via

the
index.

See Section 18.3

Drawback

•  fail
to
generated
all
the
possible
ambiguous

representatons
arising
from
the
quan(ﬁer

scope
ambigui(es.

àunderspeciﬁca(on
=
Including
all
possible

readings
without
enumera(ng
them

(probabili(es?)

46

Idioms
and
Composi(onality
(Sect
18.6)

•  What
kind
of
meaning
representa(on
do
we

need
for
idioms?

•  The
(p
of
the
iceberg
à
ﬂexible

– iceberg’s
(p

– (p
of
an
iceberg

– (p
of
a
rather
large
iceberg

– (p
of
a
larger
iceberg

•  Kick
the
bucket
à
crystallized

47

CORPUS-‐BASED
APPROACHES
AND

MACHINE
LEARNING

48

Latent
Seman(c
Analysis

(wikipedia)

http://en.wikipedia.org/wiki/Latent_semantic_analysis

•  Latent
seman(c
analysis
(LSA)
is
a
technique
of
analyzing
rela(onships

between
a
set
of
documents
and
the
terms
they
contain
by
producing
a
set

of
concepts
related
to
the
documents
and
terms.

•  LSA
assumes
that
words
that
are
close
in
meaning
will
occur
in
similar

pieces
of
text.

•  A
matrix
containing
word
counts
per
paragraph
is
constructed
from
a
large

piece
of
text
and
a
mathema(cal
technique
called
singular
value

decomposi(on
(SVD)
is
used
to
reduce
the
number
of
rows
while

preserving
the
similarity
structure
among
columns.

•  Words
are
then
compared
.
Values
close
to
1
represent
very
similar
words

while
values
close
to
0
represent
very
dissimilar
words.”

Applica$ons
and
Limita$ons…
49

Topic
Models

(wikipedia)

http://en.wikipedia.org/wiki/Topic_model

”
a
topic
model
is
a
type
of
sta(s(cal
model
for
discovering
the

abstract
"topics"
that
occur
in
a
collec(on
of
documents.
Intui(vely,

given
that
a
document
is
about
a
par(cular
topic,
one
would
expect

par(cular
words
to
appear
in
the
document
more
or
less
frequently:

"dog"
and
"bone"
will
appear
more
oWen
in
documents
about
dogs,

"cat"
and
"meow"
will
appear
in
documents
about
cats,
and
"the"
and

"is"
will
appear
equally
in
both.
A
document
typically
concerns

mul(ple
topics
in
diﬀerent
propor(ons;
thus,
in
a
document
that
is

10%
about
cats
and
90%
about
dogs,
there
would
probably
be
about
9

(mes
more
dog
words
than
cat
words.
A
topic
model
captures
this

intui(on
in
a
mathema(cal
framework,
which
allows
examining
a
set

of
documents
and
discovering,
based
on
the
sta(s(cs
of
the
words
in

each,
what
the
topics
might
be
and
what
each
document's
balance
of

topics
is.”

Latent
Dirilecht
Alloca$on
(LDA)

50

Distribu(onal
Seman(cs

(wikipedia)

http://en.wikipedia.org/wiki/Distributional_semantics

”Distribu$onal
seman$cs
is
a
research
area
that
develops
and

studies
theories
and
methods
for
quan(fying
and
categorizing

seman(c
similari(es
between
linguis(c
items
based
on
their

distribu(onal
proper(es
in
large
samples
of
language
data.
The

basic
idea
of
distribu(onal
seman(cs
can
be
summed
up
in
the

so-‐called
Distribu(onal
hypothesis:
linguis&c
items
with
similar

distribu&ons
have
similar
meanings”

Applica$ons
and
Limita$ons…

51

SemEval

(wikipedia)

http://en.wikipedia.org/wiki/SemEval

•  SemEval
(Seman(c
Evalua(on)
is
an
ongoing
series
of
evalua(ons
of

computa(onal
seman(c
analysis
systems;
it
evolved
from
the
Senseval

word
sense
evalua(on
series.
The
evalua(ons
are
intended
to
explore
the

nature
of
meaning
in
language.
While
meaning
is
intui(ve
to
humans,

transferring
those
intui(ons
to
computa(onal
analysis
has
proved

elusive.This
series
of
evalua(ons
is
providing
a
mechanism
to
characterize

in
more
precise
terms
exactly
what
is
necessary
to
compute
in
meaning.

As
such,
the
evalua(ons
provide
an
emergent
mechanism
to
iden(fy
the

problems
and
solu(ons
for
computa(ons
with
meaning.
These
exercises

have
evolved
to
ar(culate
more
of
the
dimensions
that
are
involved
in
our

use
of
language.
They
began
with
apparently
simple
afempts
to
iden(fy

word
senses
computa(onally.
They
have
evolved
to
inves(gate
the

interrela(onships
among
the
elements
in
a
sentence
(e.g.,
seman(c
role

labeling),
rela(ons
between
sentences
(e.g.,
coreference),
and
the
nature

of
what
we
are
saying
(seman(c
rela(ons
and
sen(ment
analysis).

52

In
this
course…

•  We
are
not
going
to
focus
on

formalisms
or
on
corpus-‐based

approaches
to
seman(cs.
We
will

focus
some
speciﬁc
aspects
of

meaning
that
are
useful
for
NLP

and
IR
applica(ons,
namely…

53

The
End

54

Lecture 2: Computational Semantics

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (12)

Similar to Lecture 2: Computational Semantics

Similar to Lecture 2: Computational Semantics (20)

More from Marina Santini

More from Marina Santini (20)

Recently uploaded

Recently uploaded (20)

Lecture 2: Computational Semantics