Creating own language made easy

Creating own language
made easy
Ingvar Stepanyan
@RReverser

Everything sucks – human version

Everything sucks – developer version

Not so scary magic

JS

JS

JS
Parser
Generator

Parser generators


jison Bison in javascript, used by Coffeescript



PEG.js parser generator for JavaScript based on the parsing expression grammar formalism



OMeta/JS (source) metacompiler for many languages to many targets, including js.



languagejs - PEG parser written in JavaScript with first class errors



Canopy Self-hosting PEG parser compiler for JavaScript, influenced by Ruby parser generators such as Treetop and
Citrus



JS/CC LALR(1) parser generator



jsparse PEG by Grandmaster Chris Double



ReParse parser combinator library for Javascript like Haskell's Parsec



p4js Monadic parser library for JavaScript



JSGLR Scannerless, Generalized Left-to-right Rightmost (SGLR) derivation parser for JavaScript



antlr has a javascript target



Cruiser.Parse LL(k) parser

Parser generators
Top-down (Jison)

Bottom-up (PEG.js)
start
= additive

%left "+"
%left "*"

additive
= left:multiplicative "+" right:additive { return
left + right; }
/ multiplicative

%start program;

multiplicative
= left:primary "*" right:multiplicative { return
left * right; }
/ primary

program
: expression { return $$ }
;

primary
= integer
/ "(" additive:additive ")" { return additive; }
integer "integer"
= digits:[0-9]+ { return parseInt(digits.join(""),
10); }

%%

expression
: expression "+" expression -> $1 + $3
| expression "*" expression -> $1 * $3
| "(" expression ")" -> $2
| NUMBER -> $1
;

Choice ordering
Top-down (Jison)

Bottom-up (PEG.js)
start
= additive

%left "+"
%left "*"

additive
left + right; }
/ multiplicative

%start program;

multiplicative
left * right; }
/ primary

program
;

primary
= integer
integer "integer"
10); }

%%

expression
| "(" expression ")" -> $2
| NUMBER -> $1
;

Ambiguity
if a then (if b then s) else s2
or
if a then if b then s else s2:
if a then (if b then s else s2)

Top-down (Jison)

Bottom-up (PEG.js)
start
= additive

%left "+"
%left "*"

additive
left + right; }
/ multiplicative

%start program;

multiplicative
left * right; }
/ primary

program
;

primary
= integer
integer "integer"
10); }

%%

expression
| "(" expression ")" -> $2
| NUMBER -> $1
;

Left recursion
x=1-2-3
Top-down (Jison)

Bottom-up (PEG.js)
start
= additive

%left "+"
%left "*"

additive
left + right; }
/ multiplicative

%start program;

multiplicative
left * right; }
/ primary

program
;

primary
= integer
integer "integer"
10); }

%%

expression
| "(" expression ")" -> $2
| NUMBER -> $1
;

Left recursion
x=1-2-3
Top-down (Jison)

Bottom-up (PEG.js)
[

[

"x",
"=",
[
[
"1",
"-",
"2"
],
"-",
"3"
]

"x",
"=",
[
"1",
"-",
[
"2",
"-",
"3"
]
]
]

]

Summary choice
Top-down (Jison)

Bottom-up (PEG.js)
start
= additive

%left "+"
%left "*"

additive
left + right; }
/ multiplicative

%start program;

multiplicative
left * right; }
/ primary

program
;

primary
= integer
integer "integer"
10); }

%%

expression
| "(" expression ")" -> $2
| NUMBER -> $1
;

Jison syntax: helpers
%{

var scope = {};
%}
…

Jison syntax: lexer
…

%lex
%%
s+

/* skip whitespace */

[A-Za-z_]w+

return 'ID';

d+

return ‘NUMBER’;

[+*;=]

return yytext;

<<EOF>>

return 'EOF';

/lex
…

Jison syntax: operator precedence
…

%left ';‘
%right ‘=‘
%left ‘+’
%left ‘*’

…
/*
“x=a*b+c”
-> assign(“x”, “a*b+c”)
-> assign(“x”, add(“a*b”, “c”))
-> assign(“x”, add(mul(“a”, “b”), “c”))
*/

Jison syntax: rules
%start program
program
: stmt* EOF { return $1 }
;
stmt
: expr ‘;’ -> $1
;
expr
| NUMBER -> $1
;

Methods
 string concatenation
 building AST object + escodegen (http://github.com/Constellation/escodegen)
 using SourceNode from source-map (https://github.com/mozilla/source-map)

SourceNode
 new SourceNode(line, column, filename, jsChunk)
 line, column – position in original file
 filename – name of original file
 jsChunk – JavaScript code string, another SourceNode instance or array of those

Resulting stack

JS

JS

JS
Jison
sourcemap

Creating own language made easy

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Creating own language made easy

Similaire à Creating own language made easy (20)

Plus de Ingvar Stepanyan

Plus de Ingvar Stepanyan (8)

Dernier

Dernier (20)

Creating own language made easy