1. About what cannot be said.
Maarten Marx
University of Amsterdam
May 15, 2014
2. Abstract
It is not easy to think of things that you cannot express. Language seems
limitless. On the other hand, we often use expressions like ”I cannot find
the words to thank you/say how much I love you/express my sadness” etc.
For artificial languages, it is often possible to give semantic
characterizations of their expressive power. We can use these to show that
some concept is not expressible. This is a powerful tool. Without it, we
can only say that until now we did not find a way to express the concept in
the language.But we do not know if this is because of the limits of our
knowledge or because of the limits of the language. In this presentation we
make these notions precise and show with examples how they can be used
in concrete situations.
3. Outline
1 Natural Languages
2 Expressing things succinctly
Numbers
Still used in some Georgian shops: tallying
Roman Numbers
What we all know and use: decimal system
Numbers in computers: zeros and ones
Propositional logic
Basis of all reasoning
3 basic types of sentences
Spotting contradictions
3 Limits of expressivity
First order logic and numbers
Integers and Rationals
Rationals and Reals
4 Take home message
5. Natural Languages
For natural languages, linguists have given bounds on the expressivity
of the morphology and the grammar.
E.g., Chomsky claimed that English morphology can be generated by
a regular language and English grammar by a context-free language.
Chomsky has/had the idea that people are born with some kind of
language or grammar-instinct.
But then grammars cannot be arbitrarily difficult/expressive.
It is a sport among linguists to find counterexamples to proposed
bounds.
7. Expressing things succinctly
Our aim
Show that different languages can express the same, but using less
resources.
Less resources??? That just means ”easier or shorter”
Our method
We first look at representing numbers.
Then at the basis of all reasoning: propositional logic.
8. Expressing quantities
We want to create a notation for expressing positive quantities:
1, 2, 3, 4, . . .
We evaluate our notation on how much work it is to write down a
number.
9. Still used in some Georgian shops: tallying
Definition: tallying
I, II, III, . . . , IIII, IIII I, . . . , IIII IIII
Evaluation
Clearly works, every number can be expressed
Uses really only one symbol: I
Size of the expression equals the quantity which is expressed
This is not handy for very rich people. . .
10. Roman Numbers
Definition
I, II, III, IIII, IV , . . . , IX, . . . , D, . . . , C, . . . , L, . . . , M
Evaluation
Clearly works, every number can be expressed
Uses several different symbols
Value of a symbol depends on its position: compare eg IX and XI.
Size of the expression much smaller than the quantity which is
expressed
except for really large numbers (as M is the ”largest” symbol)
11. Decimal system
Definition
1, 2, 3, . . . , 10, 11, . . . , 100, . . . , 1.000, . . . , 1.000.000, . . .
Evaluation
Uses 10 different symbols
Value of a symbol depends on its position:
In 538,
the 5 means 500,
the 3 means 30,
and the 8 means 8
To express a number n we need less than log10(n) + 1 digits.
For example, we express 1000 with 4 digits.
To express the enormous wealth of Ivanishvili we need only 10 or 11
digits.
Very handy and efficient
12. Numbers in computers: zeros and ones
Definition
1, 10, 11, 100, 101, 110, 111, 1000, . . .
Evaluation
Used in every computer.
Value of a symbol depends on its position: In 101 stands the first 1
for 22 and the last 1 for 20, and thus 101 means five.
With tallying, we used 1 symbol. In our decimal system we use 10
symbols. Here we use just 2 symbols.
Question Is the zero-one system more like tallying or more like the
decimal system in terms of the size of the expressions?
13. Answer: more like the decimal system
Number of inhabitants of Georgia
Tallying: IIII IIII IIII IIII . . . IIII IIII IIII IIII IIII IIII (very very big)
Decimal: 4.000.000
Binary string with length at most log2(4.000.000), which is
log2(4 ∗ 1.000 ∗ 1.000) = log2(4) + log2(1.000) + log2(1.000) =
2 + 10 + 10 = 22
Number of inhabitants of the world
Tallying: 8 billion: 2.000 times as long as that for Georgia
Decimal: 8 billion uses only 10 digits, about 1.5 times the number of
digits for Georgia
Binary log2(Georgia) + log2(2.000) = 22 + 11 = 33. Also just about
1.5 times longer.
14. Conclusion
Different ways of expressing the same notions
can differ very much in the ease or costs needed to express something.
15. Propositional logic
Our aim
Show that checking whether you state something that makes sense can be
very time consuming.
Our method
Employ elementary reasoning
16. Propositional logic: Basis of all reasoning
Atomic propositions
”It is hot.”
”This talk is too long.”
Represent them by symbols p, q, r, . . ..
Can be true or false. They are atoms. We do not look inside.
Connectives
not φ, φ and ψ, φ or ψ
Examples
p and not q
not ( p or q )
p or not (q and p)
p and p
17. 3 basic types of sentences
Tautology
Is always true. Example
18. 3 basic types of sentences
Tautology
Is always true. Example p or not p
Contingency
Can be true and can be false. Example
19. 3 basic types of sentences
Tautology
Is always true. Example p or not p
Contingency
Can be true and can be false. Example p or q
Contradiction
Is always false. Example
20. 3 basic types of sentences
Tautology
Is always true. Example p or not p
Contingency
Can be true and can be false. Example p or q
Contradiction
Is always false. Example p and not p
22. Ex falso quodlibet
We better be careful with stating something, because,
when we state a contradiction,
everything follows from it.
23. Example
Assume you have as true
A This talk is great
B This talk is not great
We show that ”Santa Claus exists” follows.
from A we can derive
C This talk is great or Santa Claus exists.
Combining B and C we get,
D Santa Claus exists.
Don’t worry, we can also deduce that Santa Clause does not exist.
26. Easy when formula justs lists possible worlds
Special shape
not only in front of atoms
formula is a disjunction of conjunctions
(p and q and not r) or ( p and q and not p) or (not q and . . . ) or . . .
Spotting contradictions
Is very easy and quick:
for each disjunct, check if it contains an atom and its negation;
if all are like that, the formula is a contradiction
otherwise it is not.
How long does that take?
We just walk through the formula from left to right
and store at most one disjunct in our memory.
27. But life is not always as simple
Rewrite formulas
Every formula φ can be rewritten into this special form,
without changing its meaning.
φ is thus a contradiction precisely if its special form is one.
Rewriting is expensive
In bad cases, the equivalent of φ in special form
is exponentially larger than φ.
Example: a φ of 400 symbols leads then to a special form of 2400
symbols.
This is very much. A gigabyte is 230 bytes. A terrabyte is 240 bytes.
A petabyte 250
2400 is roughly a 1 with 120 zeros, which is roughly the number of
atoms in the entire universe. . .
28. What to learn from this all
Spotting contradictions is hard
In eassence, nobody knows if there is a truely better way of spotting
contradictions than the one just sketched.
Artificial Intelligence is all about trying to do things faster in special
cases.
Expressivity
Different languages may both express the same thing
but one can be much more efficient/handy/succinct than the other
No we will look at real limits of expressive power.
29. Showing that a language has limits
Can express and cannot express
In general it is harder to show that you (and nobody else too) cannot
express something in a language
than to show that you can
in the latter case, you just do it
Example
Express ”exclusive or” in propositional logic.
p xor q is true precisly if exactly one of p and q is true
30. Showing that a language has limits
Can express and cannot express
In general it is harder to show that you (and nobody else too) cannot
express something in a language
than to show that you can
in the latter case, you just do it
Example
Express ”exclusive or” in propositional logic.
p xor q is true precisly if exactly one of p and q is true
(p and not q) or (not p and q)
31. First order logic and numbers
Talk about individuals and relations among them
Add constants, variables, quantifiers, and relations to propositional
logic.
Example
There is no largest number
32. First order logic and numbers
Talk about individuals and relations among them
Add constants, variables, quantifiers, and relations to propositional
logic.
Example
There is no largest number
not∃x∀y. y ≤ x
Every donkey has a tail
33. First order logic and numbers
Talk about individuals and relations among them
Add constants, variables, quantifiers, and relations to propositional
logic.
Example
There is no largest number
not∃x∀y. y ≤ x
Every donkey has a tail
∀ x. if DONKEY(x) then ∃ y. TAIL(x,y).
This wants Facebook: all friends of your friends are your own friends
if x Friend y and y Friend z then x Friend z.
34. Integers and Rationals
Integers Z: 1,2,3,4,. . . . Rationals Q: . . . , 1
2, . . . , 2
3, . . . , 1, . . .
We speak about them in First Order Logic using only the relation <
(smaller than).
Question: can we express the difference between integers and
rationals in FOL?
Yes!
35. Integers and Rationals
Integers Z: 1,2,3,4,. . . . Rationals Q: . . . , 1
2, . . . , 2
3, . . . , 1, . . .
We speak about them in First Order Logic using only the relation <
(smaller than).
Question: can we express the difference between integers and
rationals in FOL?
Yes!
Between any two different numbers there is another number.
36. Integers and Rationals
Integers Z: 1,2,3,4,. . . . Rationals Q: . . . , 1
2, . . . , 2
3, . . . , 1, . . .
We speak about them in First Order Logic using only the relation <
(smaller than).
Question: can we express the difference between integers and
rationals in FOL?
Yes!
Between any two different numbers there is another number.
∀x∀y. x < y ⇒ ∃z(x < z & z < y)
37. Difference between finite and infinite
Question
Can we express the difference between all integers and a finite number
of integers?
38. Difference between finite and infinite
Question
Can we express the difference between all integers and a finite number
of integers?
Yes
39. Difference between finite and infinite
Question
Can we express the difference between all integers and a finite number
of integers?
Yes
”there is a largest number”
∃x. not∃y. x < y
The same question but only using the equality relation.
We need to know how many elements there are in the finite set. Say n
∃x1∃x2 . . . ∃xn+1(x1 = x2&x1 = x3& . . .)
Note Formula is not general, and can be very long.
40. Rationals and Reals
Reals
Reals, R: all numbers which you can write using decimal expansion.
Thus all rationals, but also
√
2, π, . . .
Question
Can we express the difference between these two in FOL?
41. Rationals and Reals
Reals
Reals, R: all numbers which you can write using decimal expansion.
Thus all rationals, but also
√
2, π, . . .
Question
Can we express the difference between these two in FOL?
No!
To show that we need to play games.
42. Model comparison games
Rules
Two players (Spoiler and Duplicator) play on two models
Spoiler starts and picks a number in one model
Dupicator answers by picking a number in the other model.
these numbers are connected by a line.
and so on
Duplicator must always answer so that the lines do not cross.
if she cannot answer she lost, if she can continue forever she wins.
Games and Logic
Two models make the same formulas true if Duplicator can win any
game played on them.
43. What we learn from this
If you want to express the difference between R and Q you need a
language stronger than FOL.
The difference is:
in R every upward bounded set of numbers has a least upper bound.
The least upper bound of {n ∈ Q | n < π} is π, but π does not exists
in Q.
Note that we quantified over SETS, not just over elements.
44. Take home message
Expressivity of languages
Every language (artificial and natural) has limits to what it can
express.
These limits say something about the complexity of the language.
Know your limits
If you cannot express what you want in some language, then
it is either your fault, or
the ”fault” (the limits) of the language.
If you know these limits, you can either search further, or accept
reality and be in peace.
