Privatization and Disinvestment - Meaning, Objectives, Advantages and Disadva...
SDF vs Context-Free Grammars for Compiler Construction
1. Declarative Syntax Definition
pretty printing
Guido Wachsmuth
Delft
Course IN4303 2012/13
University of
Technology Compiler Construction
Challenge the future
4. Assessment
last lecture
Compare SDF with the plain formalism of context-free grammars.
Discuss additional description means provided by SDF and why
they are needed in compiler construction.
Pretty Printing 2
5. Assessment
last lecture
Compare SDF with the plain formalism of context-free grammars.
Discuss additional description means provided by SDF and why
they are needed in compiler construction.
• regular expressions
Pretty Printing 2
6. Assessment
last lecture
Compare SDF with the plain formalism of context-free grammars.
Discuss additional description means provided by SDF and why
they are needed in compiler construction.
• regular expressions
• layout
Pretty Printing 2
7. Assessment
last lecture
Compare SDF with the plain formalism of context-free grammars.
Discuss additional description means provided by SDF and why
they are needed in compiler construction.
• regular expressions
• layout
• AST constructors
Pretty Printing 2
8. Assessment
last lecture
Compare SDF with the plain formalism of context-free grammars.
Discuss additional description means provided by SDF and why
they are needed in compiler construction.
• regular expressions
• layout
• AST constructors
• follow restrictions
Pretty Printing 2
9. Assessment
last lecture
Compare SDF with the plain formalism of context-free grammars.
Discuss additional description means provided by SDF and why
they are needed in compiler construction.
• regular expressions
• layout
• AST constructors
• follow restrictions
• annotations for associativity
Pretty Printing 2
10. Assessment
last lecture
Compare SDF with the plain formalism of context-free grammars.
Discuss additional description means provided by SDF and why
they are needed in compiler construction.
• regular expressions
• layout
• AST constructors
• follow restrictions
• annotations for associativity
• priorities
Pretty Printing 2
13. Overview
today’s lecture
plagues of traditional parsing algorithms
• paradise lost
• paradise regained
pretty-printing
• from trees to text
• box layout in pretty-print tables
Lexical Analysis 3
14. Overview
today’s lecture
plagues of traditional parsing algorithms
• paradise lost
• paradise regained
pretty-printing
• from trees to text
• box layout in pretty-print tables
template language
• alternative to SDF
• generation of pretty-printing strategies
• generation of completion templates
Lexical Analysis 3
159. public boolean
authenticate(String user, String pw) {
SQL stm = <| SELECT id FROM Users
WHERE name = ${user}
AND password = ${pw} |>;
return executeQuery(stm).size() != 0;
}
191. public boolean
authenticate(String user, String pw) {
SQL stm = <| SELECT id FROM Users
WHERE name = ${user}
AND password = ${pw} |>;
return executeQuery(stm).size() != 0;
}
210. Recap: Parsing & AST construction
let
function fact(n : int): int = if n < 1 then 1 else n * fact(n - 1)
in
printint(fact(10))
end
Let(
[ FunDec(
"fact"
, [FArg("n", Tid("int"))]
, Tid("int")
, IfThenElse(
Lt(Var("n"), Int("1"))
, Int("1")
, Times(Var("n"), Call("fact", [Minus(Var("n"), Int("1"))]))
)
)
]
, [Call("printint", [Call("fact", [Int("10")])])]
)
Pretty Printing 183
211. Pretty Printing
from ASTs to text
• keywords
• layout: spaces, line breaks, indentation
specification
• partially defined in grammar
• missing layout
Pretty Printing 184
229. Literature
learn more
declarative syntax definition
Lennart C. L. Kats, Eelco Visser, Guido Wachsmuth: Pure and Declarative
Syntax Definition - Paradise Lost and Regained. SPLASH 2010
Pretty Printing 198
230. Literature
learn more
declarative syntax definition
Lennart C. L. Kats, Eelco Visser, Guido Wachsmuth: Pure and Declarative
Syntax Definition - Paradise Lost and Regained. SPLASH 2010
pretty-printing
Marc G.J. van den Brand, Eelco Visser: Generation of Formatters for Context-
Free Languages. ACM TOSEM 5(1):1-41, 1996.
Merijn de Jonge. Pretty-Printing for Software Reengineering. ICSM 2002
Pretty Printing 198
231. Literature
learn more
declarative syntax definition
Lennart C. L. Kats, Eelco Visser, Guido Wachsmuth: Pure and Declarative
Syntax Definition - Paradise Lost and Regained. SPLASH 2010
pretty-printing
Marc G.J. van den Brand, Eelco Visser: Generation of Formatters for Context-
Free Languages. ACM TOSEM 5(1):1-41, 1996.
Merijn de Jonge. Pretty-Printing for Software Reengineering. ICSM 2002
template language
Tobi Vollebregt, Lennart C. L. Kats, Eelco Visser. Declarative Specification of
Template-Based Textual Editors. LDTA 2012
Pretty Printing 198
232. Outlook
beyond IN4303
Model-driven Software Development
• domain-specific languages
• build your own language with Spoofax
Seminar Metaprogramming
• science behind the scenes
Master theses
• theory & practice
• WebDSL & mobl
• Spoofax
Pretty Printing 199
233. Outlook
coming next
declarative semantics definition
• Lecture 4: Term Rewriting
• Lecture 5: Static Analysis and Error Checking
• Lecture 6: Code Generation
Labs
• Sep 27 syntax definition
• Oct 4 code completion & pretty printing
Pretty Printing 200
236. Pictures
attribution & copyrights
Slide 1:
Latin Bible by Gerard Brils (photo by Adrian Pingstone), public domain
Slide 188:
West Cornwall Pasty Co. by Dominica Williamson, some rights reserved
Slide 199-200:
Ostsee by Mario Thiel, some rights reserved
Slides 5-180:
Pure and Declarative Syntax Definition: Paradise Lost and Regained, some rights reserved
Pretty Printing 203
Notes de l'éditeur
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
My name is Guido Wachsmuth, \n
and I will tell you a sad story.\n
A story about the paradise of pure and declarative syntax definition,\n
about how this paradise once was lost,\n
and about how it was regained later,\n
but also about how this paradise is denied until these days. \n
In the beginning\n
were the words, and the words\n
were trees, and the trees were words. \n
All words were made through grammars, and without grammars was not any word made that was made. \n
Those were the days of the garden of Eden. \nAnd there where language engineers strolling through the garden. \n
They made languages which were sets of words by making grammars full of beauty. \n
And with these grammars, they turned words into trees and trees into words. \n
And the trees were natural, \n
and pure, \n
and beautiful, as were the grammars.\n
Among them were software engineers who made software as the language engineers made languages. And they dwelt with them and they were one people. The language engineers were software engineers and the software engineers were language engineers. \n
And the language engineers made language software. They made recognisers to know words, and generators to make words, and parsers to turn words into trees, and formatters to turn trees into words.\n
But the software they made was not as natural, \n
and pure, \n
and beautiful as the grammars they made. \n
So they made software to make language software and began to make language software by making syntax definitions. And the syntax definitions were grammars and grammars were syntax definitions. With their software, they turned syntax definitions into language software. And the syntax definitions were language software and language software were syntax definitions.\n
And the syntax definitions were natural, \n
and pure, \n
and beautiful, as were the grammars.\n
Now the serpent was more crafty than any other beast of the field. He said to the language engineers\n\nDid you actually decide not to build any parsers?\n\nAnd the language engineers said to the serpent,\n\nWe build parsers, but we decided not to build others than general parsers, nor shall we try it, lest we lose our syntax definitions to be natural, and pure, and beautiful.\n\nBut the serpent said to the language engineers,\n\nYou will not surely lose your syntax definitions to be natural, and pure, and beautiful. For you know that when you build particular parsers your benchmarks will be improved, and your parsers will be the best, running fast and efficient.\n
So when the language engineers saw that restricted parsers were good for efficiency, and that they were a delight to the benchmarks, they made software to make efficient parsers and began to make efficient parsers by making parser definitions.\n
Those days, the language engineers went out from the garden of Eden. \n
In pain they made parser definitions all the days of their life. But the parser definitions were not grammars and grammars were not parser definitions.\n
And by the sweat of their faces they turned parser definitions into efficient parsers. \n
But the parser definitions were not natural, \n
nor pure, \n
nor beautiful, as the grammars had been before.\n
Their software was full of plagues. \n
The first plague were grammar classes. Only few grammars could be turned directly into parser definitions. And language engineers massaged their grammars all the days of their life to make them fit into a grammar class. And the parser definitions became unnatural, and impure, and ugly. And there was weeping and mourning.\n
The second plague was disambiguation. Their new parsers were deterministic. So the language engineers encoded precedence in parser definitions. And the parser definitions became unnatural, and impure, and ugly.\n\n
The third plague was lexical syntax. The new software could not handle lexical syntax definitions. So the language engineers made another software to turn lexical syntax definitions into scanners. But lexical syntax definitions were less expressive than the grammars they used before. And they were separated from parser definitions, as were scan- ners from parsers. And there was weeping and wailing.\n\n
The fourth plague was tree construction. The language engineers wanted the efficient parsers to turn words into trees, as their old parsers did. So they added code to their parser definitions. And the parser definitions became unnatural, and impure, and ugly. And those who were oblivious to the working of the efficient parsers made parsers that turn the right words into the wrong trees.\n\n
The fifth and sixth plague were evolution\n
and composition. Once the language engineers added a new rule to their parser definitions, those tended to break. And they massaged them by the sweat of their faces to make them fit again into the grammar class. And they were not able to compose two parser definitions to a single parser definition because of grammar classes and separate scanners. And there was weeping and groaning.\n
The seventh plague was the restriction to parsers. The language engineers turned parser definitions into recognizers and into parsers. But they could not turn them into generators or formatters. That was because parser definitions were not grammars.\n
Many have undertaken to compile a narrative of the things that have been accomplished among us. It seemed good to us also, having followed all things closely for some time past, to tell an orderly account for you that you may have certainty concerning the things you have been taught.\n
It all starts with the beauty of grammars and trees\n
and it all starts with Noam Chomsky.\n\nLike other linguists, he was thinking about the infinite productivity of language.\nWe can produce sentences that no-one produced before.\n\n
But how can we put this productivity into finite models?\nChomsky&#x2019;s came up with grammars as generative devices.\nFinite models, declaring how words are made from letters and sentences are made from words.\n\n
This is a grammar for generating binary numbers.\n
It consists of 4 production rules\n
At the RHS of these rules we see terminal symbols.\nThese are the symbols binary numbers are made of.\n
At the LHS of each rule we see a nonterminal symbol.\nNonterminal symbols are variables in the generation.\nThey can also occur on the right-hand side of a production rule.\nWe say: The LHS symbol produces the symbols on the RHS.\n
The start symbol is a particular nonterminal symbol.\n
When we generate words through grammars,\n
we start with this start symbol.\n
and apply production rules by replacing nonterminal symbols.\nWe replace a nonterminal symbol from the LHS of a rule with the symbols of the RHS.\n
We do this as long as there are nonterminal symbols for which we can find production rules.\nWe are free to choose any nonterminal symbol and apply any rule.\nSo let&#x2019;s take another rule for NUM this time.\n
Now we can replace one of the both symbols for digits.\nLet&#x2019;s apply the first rule.\n
There is still one nonterminal symbol left.\nLet&#x2019;s take the other rule for digits this time.\nWe are done. \nWe generated the word 10.\n
But with grammars we can not only generate words but also sentences.\n
Here is a grammar for generating expressions.\nAgain we have production rules\n
with terminal symbols occuring on the RHS of these rules.\nHere, we have two different kinds of terminal symbols:\n+ and * each representing a particular morphem,\nand NUM, representing a whole class of morphems, let&#x2019;s assume in this case decimal numbers.\n
We have only a single nonterminal symbol\n
which is also the start symbol.\n
We can now generate an expression.\n
We replace the start symbol by applying the first rule for EXP.\n
Now let&#x2019;s replace the first EXP and let&#x2019;s use the second rule this time.\n
Let&#x2019;s take again the first EXP and replace it by a morphem from the morphem class NUM.\n
When we do this two more times\n
we have generated an expression.\n
We can use grammars to describe languages by making a grammar that generates all the words of a given language.\n
For example, we can come up with a grammar for Latin.\n
Which should generate all these Latin sentences.\nAnd all other correct Latin sentences.\n
But a grammar can also define a language. \nIt defines the language of sentences that it generates.\nAnd we can make new languages my making grammars.\nAnd we can ask the grammar if a sentence is in the language or not.\nThe grammar is the only truth.\n
For this, we read the production rules of a grammar as reduction rules.\nThe symbols of the RHS can be reduced to the nonterminal symbol of the LHS.\nTo emphasise this reading, we can write the rules in a reductive way, switching LHS and RHS.\n
Now let&#x2019;s see if this is a valid expression according to our grammar we&#x2019;ve seen before.\n
We apply reduction rules where ever we can, finding the LHS of a rule and replacing it with the nonterminal symbol from the RHS.\nHere we can reduce 21 to EXP.\n
The same for 7\n
and 3.\n
Now we can replace EXP*EXP by EXP\n
and finally EXP+EXP by EXP.\nIt remains the start symbol of our grammar, telling us the truth:\n3*7+21 is a valid expression.\n
We can use grammars also to turn words into trees.\n
We do so because we are not only interested if a sentence is an element of a language,\n
but also its structure. \nAs we know from primary school, \nthis structure can typically be represented as trees.\n
we saw already two readings of grammars\n
now, here we have a third one: rules as tree construction rules. \nnonterminal symbol as the root, symbols constituting the root as leaves\n
what did before the scanner, now does the parser\n
\n
\n
\n
\n
\n
\n
\n
\n
closed under composition\n
closed under composition\n
closed under composition\n
closed under composition\n
closed under composition\n
closed under composition\n
closed under composition\n
closed under composition\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
Fear not! Stand firm! See the naturalness, and the pureness, and the beauty of declarative syntax definitions, which will work for you. \nFor the parser definitions that you see today, you shall never see again.\n
Go out to the promised land! \nMake new software to make parsers and begin to make parsers by making syntax definitions again. \nLet the syntax definitions be grammars again and grammars be syntax definitions again. \nAnd the syntax definitions will be natural, and pure, and beautiful again, as will the grammars.\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
WebDSL & mobl: Flash to HTML5, query optimisation, data synchronisation, partitioning, e-learning platform\nSpoofax: debugging, integration, testing, visualisation, incremental compilation, DSLs for robotics\n
\n
round-up on every lecture\n\nwhat to take with you\n\ncheck yourself, pre- and post-paration\n