Staged Computational Modelling of Linguistic Development

From Syllables to Syntax:
Investigating Staged Linguistic Development through
Computational Modelling

Kris Jack, Chris Reed, and Annalu Waller
[kjack|chris|awaller]@computing.dundee.ac.uk

Applied Computing, University of Dundee,
Dundee, DD1 4HN, Scotland

Staged Language Acquisition
• Language acquisition is consistently described in stages
• Lexical and syntactic acquisition strategies must operate
within a unified model
• The Model
– Training Data
– Initial Assumptions
– Lexical Acquisition
– Syntactic Acquisition
– Comprehension
• Results

Holophrastic Early Multi-
Pre-linguistic Stage Late Multi-word Stage Abstract Stage
Stage word Stage

0 months 6 months 12 months 18 months 24 months 30 months 36 months 42 months

Lexical Acquisition
• Siskind
• Steels
• Regier Siskind (1996)
• Cross-situational analysis
– Relationship between the appearance
and words and their referents

Lexical Acquisition
• Siskind
• Steels
• Regier Steels (2001)
• Language games
– Social pressure to communicate
within a community of agents can
lead to an emergent and shared
vocabulary

Lexical Acquisition
• Siskind
• Steels
• Regier Regier (2005)
• Associative learning
– Fast-mapping
– Shape bias
• No mechanistic changes
– Selective attention

Syntactic Acquisition
• Roy
• Elman
• Kirby Roy (2002)
• Trained a grounded robot to play a
‘show-and-tell’ game
– Training data were divided into
Data
0>t>x simple and complex descriptions
x>t>y
Data Model

y>t>z
Data

• Roy
• Elman
• Kirby Elman (1993)
• Incremental Learning
– Mechanistic changes can lead to
changes in behaviour
t>0
Module

t>x Model
Data Module

t>y
Module

• Roy
• Elman
• Kirby Kirby (2002)
• Iterated Learning
– Languages with increasing
complexity can emerge across
generations of agents
Data Data

Model Model Model

Question
Can we develop a unified model that performs
staged language acquisition where:
1. The learning mechanisms are constant AND
2. Exposure to training data is constant?

Bridging the Gap
between Words and Syntax
• Jack, Reed, and Waller (2004)
– Shift from holophrastic to syntactic language
– The shift was unrealistic as it appeared very early
• A form of substitution was employed (similar to Harris (1966); Wolff
(1988); Kirby (2002); van Zaanen (2002))
• If the model encountered A B and A C then B and C were considered
substitutable for one another
– Given the two rules:
» S/eats(john, cake) → johneatscake
» S/eats(mary, cake) → maryeatscake
– Three rules were derived:
» S/eats(x, cake) → N/x eatscake
» N/john → john
» N/mary → mary
• This is a reasonable, yet powerful, form of syntactic learning
– The target language was unrealistically simple (two-word sentences)

Training Data
• Played the Scene Building Game
– Based on the Miniature Language Acquisition
Problem (Feldman et al., 1990)
– Aim; describe a visual event so that someone else
can recreate the event based on the description

→ → →
t=1 t=2 t=3 t=4

Training Data
a red square has
appeared

→ → →
t=1 t=2 t=3 t=4

Training Data
a pink cross to the
upper right of the
red circle

→ → →
t=1 t=2 t=3 t=4

Training Data
– Aim; describe a visual event socross onsomeone else
a blue that
can recreate the event based otherthe of
the on side description
the red circle

→ → →
t=1 t=2 t=3 t=4

Training Data
another red circle
under the pink
cross

→ → →
t=1 t=2 t=3 t=4

Initial Assumptions
• Joint attention is established at around one-year-old
(Tomasello, 1995)
• Receives <event, description> pairs
– An event is a set of six feature tuples
– A description is a string

{<red, (1)>, <circle, (1)>, <pink, (2)>, <cross, (2)>, <above, (0)>, <right, (0)>}

→
t=1 t=2
“a pink cross to the u pper right of the red cir cle”

Initial Assumptions
• Sensitivity to data
– Children can identify objects through displacement during
motion (Kellman et al., 1987).
– Children can use shape and colour to differentiate between
objects (e.g. Landau et al., 1988)

→
t=1 t=2

Initial Assumptions
– Children show sensitivity to the relative spatial
relationships between objects, making distinctions between
left and right, and above and below (Quinn, 2003)


→
t=1 t=2

Initial Assumptions
– Children can perform analogies (Gentner and Medina,
1998)


→
t=1 t=2

Initial Assumptions
– Children can determine transitional probabilities between
syllables (Saffran, Aslin, and Newport, 1996)


→
t=1 t=2

The Model
• Training the model
– The Lexical Analysis Unit
• Discovers string-meaning associations
– The Syntactic Analysis Unit
• Discovers compositional relationships

The Lexical Analysis Unit
<event, description> pairs are compared through
a form of cross-situational analysis

<event, description>#1

{<green, (1)>, <circle, (1)>, <red, (2)>, <diamond, (2)>, <even_vertical, (0)>, <right, (0)>}
“a red dia mond to the right of the green cir cle”

Feature tuple comparisons are value sensitive and object
identifier insensitive. Two feature tuples, <v1, (o1)>
and <v2, (o2)>, are equivalent iff v1 = v2



Co-occurring syllable sequences are found



New <feature tuple set, description> pairs are
derived

<{<red, (1)>, <circle, (1)>, <right, (0)>}, “a”> <{<circle, (1)>, <red, (2)>, <right, (0)>}, “a”>
<{<red, (1)>, <circle, (1)>, <right, (0)>}, “to the”> <{<circle, (1)>, <red, (2)>, <right, (0)>}, “red”>
<{<red, (1)>, <circle, (1)>, <right, (0)>}, “right of <{<circle, (1)>, <red, (2)>, <right, (0)>}, “to the”>
the”> <{<circle, (1)>, <red, (2)>, <right, (0)>}, “right of
<{<red, (1)>, <circle, (1)>, <right, (0)>}, “red”> the”>
<{<red, (1)>, <circle, (1)>, <right, (0)>}, “cir cle”> <{<circle, (1)>, <red, (2)>, <right, (0)>}, “cir cle”>

• Cross-situational analysis can produce pairs that share
the same strings (homonyms) or the same feature sets
(synonyms)
• Homonyms and synonyms are removed, following a
principle of mutual exclusivity (Markman and
Wachtel, 1988)
• When pairs are equal, with insensitivity to object
identifiers, they are merged. Merging produces a new
pair, that expresses both of the relationships
<{<red, (1)>}, “red”> is merged with
<{<red, (2)>}, “red”> to produce
<{<red, (1, 2)>}, “red”>

• From all merged pairs, homonyms are removed by
selecting the most probable feature set for each
string, Frequency of (Sj | Fi )
P(Fi | Sj ) =
Frequency of Sj
where Frequency of (Sj | Fi) is the number of times that Sj has been
observed with Fi and the Frequency of Sj is the number of times that Sj
has been observed
• Then synonyms are removed by selecting the most
probable string for each feature set, P(Sj | Fi), and
erasing the remaining pair’s feature sets
• A set of lexical items are derived

The Syntactic Analysis Unit
• Compositional relationships are found by
combining and comparing lexical items
• Lexical items are combined by set union and
string concatenation
f 1, s1 combined with f 2, s 2 = f 1  f 2, s1 + s 2

• The lexical item triple <<f1, s1>, <f2, s2>, <f3, s3>>
expresses a compositional relationship iff
<f1, s1> = <f2, s2> combined with <f3, s3>

A lexical item triple can be made to express a
rule by:
1. Converting lexical items into phrasal categories
2. Constructing transformations

rule by:
<<{<red, (1, 2)>, <square, (1, 2)>}, “red square”>,
<{<red, (1, 2)>}, “red”>, <{<square, (1, 2)>}, “square”>>

rule by:

<{<red, (1, 2)>, <square, (1, 2)>}, “red square”>

<{<red, (1, 2)>}, “red”> <{<square, (1, 2)>}, “square”>

rule by:

<{<red, (1, 2)>, <square, (1, 2)>}, “red square”>
(<(1, 2) → (1, 2)>) (<(1, 2) → (1, 2)>)

<{<red, (1, 2)>}, “red”> <{<square, (1, 2)>}, “square”>

Rules that modify object identifiers can be
constructed
<{<red, (2)>}, “a red”>
() (<(1, 2) → (2)>)

<{}, “a”> <{<red, (1, 2)>}, “red”>

<{<blue, (1)>}, “the blue”>
() (<(1, 2) → (1)>)

<{}, “the”> <{<blue, (1, 2)>}, “blue”>

Rules can be merged when they share
transformations <{<red, (1, 2)>, <circle, (1, 2)>}, “red cir cle”>

<{<blue, (1, 2)>, <circle, (1, 2)>}, “blue cir cle”> (<(1, 2) → (1, 2)>) (<(1, 2) → (1, 2)>)

<{<red, (1, 2)>}, “red”> <{<circle, (1, 2)>}, “cir cle”>
(<(1, 2) → (1, 2)>) (<(1, 2) → (1, 2)>)

<{<blue, (1, 2)>}, “blue”> <{<circle, (1, 2)>}, “cir cle”>
<{<pink, (1, 2)>, <diamond, (1, 2)>}, “pink dia mond”>

(<(1, 2) → (1, 2)>) (<(1, 2) → (1, 2)>)
<{<pink, (1, 2)>}, “pink”> <{<diamond, (1, 2)>}, “dia mond”>

Rules can be merged when they share
transformations <{<red, (1, 2)>, <circle, (1, 2)>}, “red cir cle”>

<{<blue, (1, 2)>, <circle, (1, 2)>}, “blue cir cle”> (<(1, 2) → (1, 2)>) (<(1, 2) → (1, 2)>)

<{<red, (1, 2)>}, “red”> <{<circle, (1, 2)>}, “cir cle”>
(<(1, 2) → (1, 2)>) (<(1, 2) → (1, 2)>)

<{<blue, (1, 2)>}, “blue”> <{<circle, (1, 2)>}, “cir cle”>
<{<pink, (1, 2)>, <diamond, (1, 2)>}, “pink dia mond”>

(<(1, 2) → (1, 2)>) (<(1, 2) → (1, 2)>)
{<{<red, (1, 2)>, <circle, (1, 2)>}, “red cir cle”>,
<{<blue, (1, 2)>, <circle, (1, 2)>}, “blue cir cle”>, <{<pink, (1, 2)>}, “pink”> <{<diamond, (1, 2)>}, “dia mond”>
<{<pink, (1, 2)>, <diamond, (1, 2)>}, “pink dia mond”>}

(<(1, 2) → (1, 2)>) (<(1, 2) → (1, 2)>)

{<{<red, (1, 2)>}, “red”>, {<{<circle, (1, 2)>}, “cir cle”>,
<{<blue, (1, 2)>}, “blue”>, <{<diamond, (1, 2)>}, “dia mond”>}
<{<pink, (1, 2)>}, “pink”>}

Comprehension
• The model is tested for evidence of language
acquisition through comprehension tasks
• The model can comprehend a string by:
– Finding it in a phrasal category (lexical)
– Or creating it through applying a rule (syntactic)

Comprehension
• Example 1. Comprehension of “cir cle”
– Find “cir cle” in a phrasal category
– Attempt to create “cir cle” by applying a rule

<{<blue, (1, 2)>, <circle, (1, 2)>}, “blue cir cle”>,

(<(1, 2) → (1, 2)>) (<(1, 2) → (1, 2)>)

<{<pink, (1, 2)>}, “pink”>}

Comprehension


(<(1, 2) → (1, 2)>) (<(1, 2) → (1, 2)>)

<{<pink, (1, 2)>}, “pink”>}

Meaning is {<circle, (1, 2)>}

Comprehension
• Find “cir” and “cle” in phrasal categories of a rule

(<(1, 2) → (1, 2)>) (<(1, 2) → (1, 2)>)

<{<pink, (1, 2)>}, “pink”>}

No meaning found

Comprehension


(<(1, 2) → (1, 2)>) (<(1, 2) → (1, 2)>)

<{<pink, (1, 2)>}, “pink”>}

Meaning is found lexically as {<circle, (1, 2)>}

Comprehension
• Example 2. Comprehension of “red dia mond”
– Find “red dia mond” in a phrasal category
– Attempt to create “red dia mond” by applying a rule


(<(1, 2) → (1, 2)>) (<(1, 2) → (1, 2)>)

<{<pink, (1, 2)>}, “pink”>}

Comprehension


(<(1, 2) → (1, 2)>) (<(1, 2) → (1, 2)>)

<{<pink, (1, 2)>}, “pink”>}

No meaning found

Comprehension
• Find “red” and “dia mond” in phrasal categories of a rule
• Find “red dia” and “mond” in phrasal categories of a rule

(<(1, 2) → (1, 2)>) (<(1, 2) → (1, 2)>)

<{<pink, (1, 2)>}, “pink”>}

{<red, (1, 2)>}, transformed by (<(1, 2)> → (1, 2)>), and combined with
{<diamond, (1, 2)>}, transformed by (<(1, 2)> → (1, 2)>), gives {<red, (1, 2)>, <diamond, (1, 2)>}

Comprehension
• Find “red” and “dia mond” in phrasal categories of a rule
• Find “red dia” and “mond” in phrasal categories of a rule

(<(1, 2) → (1, 2)>) (<(1, 2) → (1, 2)>)

<{<pink, (1, 2)>}, “pink”>}

No meaning found

Comprehension


(<(1, 2) → (1, 2)>) (<(1, 2) → (1, 2)>)

<{<pink, (1, 2)>}, “pink”>}

Meaning is found syntactically as {<red, (1, 2)>, <diamond, (1, 2)>}

Comprehension
• Phrasal categories are substitutable for one
another if they share a subset relationship
{<{<red, (2)>}, “a red”>,
<{<blue, (2)>}, “a blue”>,
<{<pink, (2)>}, “a pink”>,
<{<green, (2)>}, “a green”>,
<{<white, (2)>}, “a white”>}

() (<(1, 2) → (2)>)

{<{}, “a”>} {<{<red, (1, 2)>}, “red”>, {<{<red, (1, 2)>, <circle, (1, 2)>}, “red cir cle”>,
<{<blue, (1, 2)>}, “blue”>, <{<blue, (1, 2)>, <circle, (1, 2)>}, “blue cir cle”>,
<{<pink, (1, 2)>}, “pink”>, <{<pink, (1, 2)>, <diamond, (1, 2)>}, “pink dia mond”>}
{<{<pink, (1)>}, “the pink”>,
<{<green, (1, 2)>}, “green”>,
<{<green, (1)>}, “the green”>,
<{<white, (1, 2)>}, “white”>} (<(1, 2) → (1, 2)>) (<(1, 2) → (1, 2)>)
<{<white, (1)>}, “the white”>}

() {<{<red, (1, 2)>}, “red”>, {<{<circle, (1, 2)>}, “cir cle”>,
(<(1, 2) → (1)>)
<{<pink, (1, 2)>}, “pink”>}
{<{}, “the”>} {<{<pink, (1, 2)>}, “pink”>,
<{<green, (1, 2)>}, “green”>,
<{<white, (1, 2)>}, “white”>}

Comprehension
{<{<red, (2)>}, “a red”>,
<{<blue, (2)>}, “a blue”>,
<{<pink, (2)>}, “a pink”>,
<{<green, (2)>}, “a green”>,
<{<white, (2)>}, “a white”>}

() (<(1, 2) → (2)>)

<{<green, (1, 2)>}, “green”>,
<{<white, (1, 2)>}, “white”>} (<(1, 2) → (1, 2)>) (<(1, 2) → (1, 2)>)

Construction Islands <{<pink, (1, 2)>}, “pink”>}

Comprehension
{<{<red, (2)>}, “a red”>,
<{<blue, (2)>}, “a blue”>,
<{<pink, (2)>}, “a pink”>,
<{<green, (2)>}, “a green”>,
<{<white, (2)>}, “a white”>}

() (<(1, 2) → (2)>)

<{<green, (1, 2)>}, “green”>,
<{<white, (1, 2)>}, “white”>} (<(1, 2) → (1, 2)>) (<(1, 2) → (1, 2)>)

<{<pink, (1, 2)>}, “pink”>,
<{<green, (1, 2)>}, “green”>,
<{<white, (1, 2)>}, “white”>}

Results
• Observing the developmental shift from lexical to syntactic comprehension
– Tested for comprehension of colours (10), shapes (10), and colour shape combinations
(100) during training. Results are averaged over 10 sessions.

Developmental Shift

25
3

3

3
3
8

Pre-linguistic

Holophrastic

20

Multi-word
Early
comprehended
No. strings

15 Lexical
10 Syntactic

5

0
0 3 6 9 12 15 18 21 24 27 30
No. <event, description>s entered

Results
• Comprehension of colours and shapes compared to colour shape
combinations
– Tested for lexical comprehension of colours (10), and shapes (10), and syntactic
comprehension of colour shape combinations (100) during training. Results are averaged
over 10 sessions.

Expressivity
f

100
% string set

80
comprehended

60 Lexical
40 Syntactic
20
0
0 5 10 15 20 25 30 35 40 45 50 55 60 65
No. <event, description>s entered

Conclusions
The model demonstrates staged linguistic
acquisition
– No maturational triggers are employed
– Training data are kept constant
– Lexical items are required before compositions can
be derived

Conclusions
The model demonstrates staged linguistic
acquisition
– No maturational triggers are employed
– Training data are kept constant
– Lexical items are required before compositions can
be derived
Can this work be extended into further stage
transitions?

Staged Computational Modelling of Linguistic Development

Recommandé

Recommandé

Contenu connexe

Similaire à Staged Computational Modelling of Linguistic Development

Similaire à Staged Computational Modelling of Linguistic Development (10)

Plus de Kris Jack

Plus de Kris Jack (17)

Dernier

Dernier (20)

Staged Computational Modelling of Linguistic Development