Combinatory Categorial Grammar

1. Combinatory
Categorial
Grammar
Jie Cao
1

2. What is CCG?(Literally)
2
Combinatory Categorial Grammar
CCG is fun
NP:
CCG
SNP/ADJ:
λf.λx.f(x)
ADJ:
fun’
>
SNP:
λx.fun’x
<
S:
fun’CCG

3. Outline
• Grammar
• Categorial
• Combinatory
3

4. Grammar Outline
• Grammar
- Phrase Structure Grammar & Dependency Grammar
- Chomsky Hierarchy (Context-free & Context Sensitive)
- Why CCG?
• Categorical
• Combinatory
4

5. Grammar Overview:
Phrase-Structure Grammar & Dependency Grammar
5
constituency relation dependency relation
sentence: CCG is fun
S
NP VP
NNP VBZ JJ
CCG is fun
fun/JJ
CCG/NNP is/VBZ
nsubj cop
CFG
S -> NP VP
NP-> NNP
VP -> VBZ JJ

6. Context-Free Grammar
6
CFG
S -> NP VP
NP-> NNP
VP -> VBZ JJ
PCFG
S -> NP VP 0.5
NP-> NNP 0.4
VP -> VBZ JJ 0.6
Lexicalized-PCFG (Head-Driven)
S(is) -> NP(CCG) VP(is)
NP(CCG)-> NNP(CCG)
VP(is) -> VBZ(is) JJ(fun)
N a set of non-terminal symbols (or variables)
Σ a set of terminal symbols (disjoint from N)
R a set of rules or productions, each of the form A → β , where A is a non- terminal, β
is a string of symbols from the infinite set of strings (Σ∪N)
S a designated start symbol
Ambiguity
4-tuples

7. Grammar Overview:
Chomsky Hierarchy
7
1. Turning-machines: http://www.slideshare.net/lavishka_anuj/turing-machines-12176328
2. Wikipedia

8. Grammar Overview:
Chomsky hierarchy
8
Categorial Grammar,CCG
[mildly Context-sensitive]
Phrase-Structure
/Constituency
[Context-free]
Nature Language Complexity
Before 1982, NL is not context free.
In 1982, Gerald Gazdar and Geoffrey Pullum have argued that despite a few non-context-free
constructions in natural language (such as cross-serial dependencies in Swiss German and
reduplication in Bambara ), the vast majority of forms in natural language are indeed context-free.
Why CCG?
0. Expressivity, support many Linguistic Phenomena
mildly Context sensitive

9. Grammar Overview:
Syntactic & Semantic
9
Why CCG?
1. Capture syntactic and semantic information jointly
2. Computable / Executable by evaluating it
CCG is fun
NP:
CCG
SNP/ADJ:
λf.λx.f(x)
ADJ:
fun’
>
SNP:
λx.fun’x
<
S:
fun’CCG

10. Summary: Why CCG?
10
Why CCG?
0. Expressivity, support many Linguistic
Phenomena, mildly context-sensitive
1. Capture syntactic and semantic information
jointly
2. Computable / Executable

11. What is CCG?(Components)
11
• Categorial Lexicon
Based on Ajdukiewicz (1935) and Bar-Hillel (1953)
Generalized (AB) Categorial Grammar
• Combinatory Logic
Based on Curry and Feys (1958).
Combinatory Categorial Grammar

12. Categorial Grammar
• Grammar
• Categorial
- Categories (Building blocks, Mapping Syntactic into Semantic))
- Atomic Categories and Complex Categories
- λ-calculus expression
- Categorial Lexicon Entry
- (pair words and phrases with meaning captured by Categories)
- Basic Function Application Rules (Forward and Backward)
• Combinatory
12

13. Categories
13
• Basic building block
• Capture syntactic and semantic information jointly
Syntactic
Category
Type
logic form:
λ-calculus expression
eta-rules: fun’
colon means interpreting the Categories

14. Categories
Atomic and Complex
14
Atomic Categories
a finite set of atomic categories, usually S,NP,N,PP… C2
Complex Categories
if X, Y 2 C, then XY, X/Y 2 C
is
CCG NP
fun ADJ:λx. fun(x)
(SNP)/ADJ:λf.λx.f x
X : Result left most
backslash : left argument
forwardslash: right argument
Y: Argument to wait for

15. 15
λx . + x 1
Categories
Example: λ-abstraction Syntax
λ means here coms a function
x is a variable: formal parameter , means λ binds to x
. separate variable with function body
+ x 1 the function body, prefix form
currying: think of all functions has a single argument only
(+ 3 4) -> ((+ 3) 4)f = +3+4
function apply to 3 and return a new function, and apply to 4
Apply λ-abstraction to arguments: an instance of λ-abstraction
(λx . + x 1) 4 -> + 4 1

16. Categories
λ-calculus expression syntax
16
<exp>:: = <constant>
| <variable>
| <exp> <exp>
| λ <variable> . <exp>
Built-in constants
Variable names
Applications
Lambda abstractions
CCG NP
fun ADJ:λx. fun(x)
is (SNP)/ADJ:λf.λx.f(x)

17. Categories
how to ‘calculate’ λ-calculus
• By Reduction
• Beta-rules (left associate )
• λx.(λy. - y x) 4 5 -> (λy. -y 4) 5 -> - 5 4 -> 1
• Alpha-rules
• (λx. +1 x) is equivalent to (λy. + 1 y), so renaming
• Eta-rules (simply)
• (λx. F x) -> F, if F denotes a function and x not occur free in F
• ADJ: λx.fun(x) -> fun’
17

18. Categorial Lexicon Entry
18
Nature
Language Categories

19. 19
Function Application Rule
Forward & Backward
• Forward Application
• X/Y: f Y:a => X: f(a)
• Backward Application
• Y:a XY:f => X: f(a)
For functor to combine with argument
A&B (“Pure”) Categorial Grammar
Determined by slash direction

20. Functional Application Rule
Example 1: CCG is fun
20
CCG is fun
NP:
CCG
SNP/ADJ:
λf.λx.f(x)
ADJ:
fun’
>
SNP:
λx.fun’x
<
S:
fun’CCG
Right to Left Order, the subject last
(Except for other rules)

21. Still Equivalent to CFG
21
constituency relation
S
NP VP
NNP VBZ JJ
CCG is fun
CFG
S -> NP VP
NP-> NNP
VP -> VBZ JJ
CCG is fun
NP:
CCG
SNP/ADJ:
λf.λx.f(x)
ADJ:
fun’
SNP:
λx.fun’x
S:
fun’CCG

22. Between CFGs and CGs
22
CFGs CGs
Combination Rules
Many Few
Parse Tree Node Non-Terminals Categories
Syntactic Symbols Few Dozen
Handful, can combined
into complex
Paired with Words POS tags Categories
Semantic Interpretation No λ-calculus

23. “Pure”CG Summary
• Grammatical categories consist of
-(a) a syntactic type defining valency (the
number and syntactic type of its arguments, if
any) and the type of the result
-(b) a logical form (Semantic)
-(c) a phonological form. (Not covered here)
• Basic “Function Application” to Combine
- Forward
- Backward
• Still Equivalent to CFG
23

24. Combinatory Categorial
Grammar
• Grammar
• Categorial
• Combinatory
- Coordination
- Composition : Combinator B
- Type Raising : Combinator T
- Substitution : Combinator S
- Principles: Control and Constrain the Power of expressive
- Others
24

25. Coordination
• Simplified coordination rule
25
< >
< >
X CONJ X’ => X’’
x,x’,x’’ are the same category type with different interpretation
• Coordination
X:g CONJ:b X: f => X:λ…b(f…)(g…)

26. Coordination
26
>B
(SNP)/NP
:λx.λy.might’(marry’x)y
(SNP)/NP
:λx.λy.and’(might’(marry’x)y)(meet’x y)
Anna marry Manny
NP
:Anna’
VP/NP:
marry'
NP:
manny’
met and
(SNP)/NP
:meet’
CONJ
:and’
might
(SNP)/VP
:might’
< >
>
SNP
:λy.and’(might’(marry’manny’)y)(meet’manny’y)
<
S
:and’(might’(marry’manny’)anna’)(meet’marry’anna’)
< >

27. Combinator B: Composition
• Forward Composition >B
• X/Y: f Y/Z:g => X/Z: λx.f(gx)
• Backward Composition <B
• YZ:g XY:f => XZ: λx.f(gx)
27

28. Composition Example
>B^2
((SNP)/PP)/NP
:λx.λy.might’(give’x)y
I give, a
NP
:I’
(VP/PP)/NP:
give'
NP:
flower’
offered, and
((SNP)/PP)/NP
:meet’
CONJ
:and’
might
(SNP)/VP
:might’
flower to police
PP:
to’police

29. Combinator T: Type Raising
29
• Forward Type-raising >T
• X:a => T/(TX): λf.f a
• where TX is a parametrically licensed category for the language
• Backward Type-raising <T
• X:a => T(T/X): λf.f a
• where TX is a parametrically licensed category for the language

30. >B
SNP
:λx.detest’x I
Anna detest Manny
NP
:Anna’
(SNP)/NP
marry'
NP:
manny’
married and
(SNP)/NP
:meet’
CONJ
:and’
I
NP
:might’
SNP
:λx.and’(marry’x Anna’)(dest’x I)
< >
Type Raising
>T
S/(SNP)
>T
S/(SNP)
>B
SNP
:λx.marry’x Anna’
30
>
S
:and’(marry’manny’anna’)(dest’manny’I’)

31. Combinator S: Substitution
31
• Forward Crossing Substitution >Sx
• (X/Y)/Z :f YZ:g => X/Z: λx.fx(g(x))
• Forward Substitution >S
• (X/Y)/Z :f Y/Z:g => X/Z: λx.fx(g(x))
• Backward Crossing Substitution <Sx
• Y/Z:g (XY)/Z :f => X/Z: λx.fx(g(x))
• Backward Substitution <S
• YZ:g (XY)Z :f => X/Z: λx.fx(g(x))

32. >B
(VPVP)/NP
:λx.λq.without’(reading’x) q
articles without reading
NP
:articles’
(VPVP)/VPing
:λp.λq.without’pq
VPing/NP:
read’
which I will
(NN)/(S/NP)
:which’
S/VP
:λy.will’ y I’
file
VP/NP
:file’
Substitution
>Sx
VP/NP
:λx.without’ (reading’ x) (file’ x )
32
>
N/N
: λx.which’ (will’ (without’ (reading’ x) (file’ x )) I’) x
>B
S/NP
:λx.will’ (without’ (reading’ x) (file’ x )) I’

33. Principles: Control Expressive
33
• The Principle of Adjacency
Combinatory rules may only apply to finitely many phonologically realized and string-adjacent
entities
• The Principle of Adjacency
All syntactic combinatory rules must be consistent with the directionality of the principal
function.
• The Principle of Inheritance
The Principle of Inheritance If the category that results from the application of a combinatory
rule is a function category, then the slash defining directionality for a given argument in that
category will be the same as the one(s) defining directionality for the corresponding
argument(s) in the input function(s).

34. Principles and Special
Phenomena
34

35. Summary: Combinatory
Coordination
Composition >B,<B
Type Raising,>T,<T
Substitution,>S,<S,<Sx,>Sx
Principles
35
< >

36. 36

37. Data and Tools
• CCG Site:
• http://groups.inf.ed.ac.uk/ccg/software.html
• Dataset Corpus
• CCGBank
• GeoQuery
• Atis
• OpenCCG etc.
37

38. CCG Parser
• Clark 2007 (The C&C Parser and Supertagger) (CKY chat parsing)
• Probabilistic CCG for Semantic Parsing(Zettlemoyer and Collins, 2005)
• For Geoquery, 96% precision, 79% recall
• Relaxed CCG for Spontaneous Speech (Zettlemoyer and Collins, 2007)
• For ATIS, 91% precision, 82% recall
• For Geoquery, 95% precision, 83% recall, Up from 79% recall
• Kwiatkowski et al., 2010
38

39. Application
Querying databases
Referring to physical objects
Executing instructions
39

40. Q&A
Thanks
40
CCG is fun
NP:
CCG
SNP/ADJ:
λf.λx.f(x)
ADJ:
fun’
>
SNP:
λx.fun’x
<
S:
fun’CCG