1. Regexes
and Grammars
in Perl 6

2. Preface

4. Synopsis 5

5. Synopsis 5
Regexes and Rules

6. S05

7. Damian Conway
Allison Randal
Patrick Michaud
Larry Wall
Moritz Lenz

8. Created: 24 Jun 2002
Last Modified: 30 Aug 2010
Version: 132

9. 54 pages

10. Part I
Regexes

11. Random facts
and terminology

12. Regular expressions
in Perl 5 were not regular

13. Regular expressions
in Perl 5 were not regular
Regular expressions
in Perl 6 are called regexes

14. Regular expressions
in Perl 5 were not regular
Regular expressions
in Perl 6 are called regexes
Which means “kinda like
a regular expression”

15. Match object
contains result of matching
$/

16. Capture variable indexes
start with 0
$0

17. $0, $1, etc.
are part of $/

18. my $q = "Hotels in Berlin";
$q ~~ /ins(.*)/;
say $0; # Berlin
say $/[0]; # Berlin

19. Metacharacters
are everything except
Unicode letters
or numbers
or underscore

20. Quotes
may be used for creating
atoms
'I will never use PHP again. '*

21. Repetition
(d+ s?) ** 3
(d+ s?) ** 5..10
d+ ** ','

22. /x modifier gone
"ab" ~~ / a b /;
say $/; # ab

23. /s, /m modifiers gone
"a1nb2nc3" ~~ /N+/;
"a1nb2nc3" ~~ /^^ .2 $$/;

24. /e modifier gone
$str =~ s/pattern/{action()}/;

25. Modifier syntax
@names =
$str =~ m:i/MiSteR s (w+)/;

26. Brackets

27. Capturing group
(...)

28. Non-capturing group
[...]

29. Character class
<[ . . . ]>

30. Embedded closure
{...}

31. Embedded closure
{...}
> "500" ~~ /(d+) {$0 < 200 or fail}/
===SORRY!===

32. Named rule or token
<. . .>

33. Part II
Grammars

34. Keywords

35. grammar
rule
token
proto
TOP

36. grammar Grammar {
rule TOP {...}
rule some_rule {...}
token some_token {...}
}

37. grammar Grammar {
rule TOP {...}
rule some_rule {...}
token some_token {...}
}
Syntax is similar
to class definition

38. grammar Grammar {
rule TOP {...}
rule some_rule {...}
token some_token {...}
}
Grammar.parse($string);

39. Example.
Step by step

40. Executed by Rakudo
rakudo.org

41. Executed by Rakudo
rakudo.org
Sometimes it fails

43. City

44. grammar SearchQuery {
}

45. grammar SearchQuery {
rule TOP {
}
}

46. grammar SearchQuery {
rule TOP {
^
$
}
}

47. grammar SearchQuery {
rule TOP {
^
<query>
$
}
}

48. grammar SearchQuery {
rule TOP {
^
<query>
$
}
}
Easy, isn't it?

49. Grammars are part
of the language

50. grammar SearchQuery {
rule TOP {
^
<query>
$
}
rule query {
}
}

51. grammar SearchQuery {
rule TOP {
^
<query>
$
}
rule query {
<city>
}
}

52. grammar SearchQuery {
rule TOP {
^
<query>
$
}
rule query {
<city>
}
token city {
}
}

53. grammar SearchQuery { N. B.
rule TOP {
^
<query>
$ rules
}
rule query {
<city>
} token
token city {
}
}

54. token
is a "word"

55. rule
is a "phrase"

56. grammar SearchQuery {
rule TOP {
^
<query>
$
}
rule query {
<city>
}
token city {
}
}

57. grammar SearchQuery {
rule TOP {
^
<query>
$
}
rule query {
<city>
}
token city {
<capital>
}
}

58. ^
<query>
$
}
rule query {
<city>
}
token city {
<capital>
}
token capital {
}
}

59. my $result = SearchQuery.parse("Amsterdam");
say $result.perl;

60. Match.new(
from => 0,
orig => "Amsterdam",
to => 9,
named => {
query => Match.new(
from => 0,
orig => "Amsterdam",
to => 9,
named => {
city => Match.new(
from => 0,
orig => "Amsterdam",
to => 9,
named => {
capital => Match.new(
from => 0,
orig => "Amsterdam",

61. Match.new(
from => 0, Matched text
orig => "Amsterdam",
to => 9,
named => {
query => Match.new(
from => 0,
orig => "Amsterdam",
to => 9,
named => {
city => Match.new(
from => 0,
orig => "Amsterdam",
to => 9,
named => {
capital => Match.new(
from => 0,
orig => "Amsterdam",

62. Match.new(
from => 0,
orig => "Amsterdam",
to => 9,
named => {
query => Match.new( rule query {
from => 0, }
orig => "Amsterdam",
to => 9,
named => {
city => Match.new(
from => 0,
orig => "Amsterdam",
to => 9,
named => {
capital => Match.new(
from => 0,
orig => "Amsterdam",

63. Match.new(
from => 0,
orig => "Amsterdam",
to => 9,
named => {
query => Match.new(
from => 0,
orig => "Amsterdam",
to => 9,
named => {
city => Match.new( token city {
from => 0, }
orig => "Amsterdam",
to => 9,
named => {
capital => Match.new(
from => 0,
orig => "Amsterdam",

64. Match.new(
from => 0,
orig => "Amsterdam",
to => 9,
named => {
query => Match.new(
from => 0,
orig => "Amsterdam",
to => 9,
named => {
city => Match.new(
from => 0,
orig => "Amsterdam",
to => 9,
named => {
capital => Match.new( token capital {
from => 0, }
orig => "Amsterdam",

65. Country

66. rule query {
<city>
| <country>
}

67. rule query {
<city>
| <country>
}
rule country {
'Afghanistan'
| 'Akrotiri'
| 'Albania'
| 'Algeria'
| 'American Samoa'
| 'Andorra' . . .
}

68. my $result = SearchQuery.parse("Amsterdam");
say $result.perl;
$result = SearchQuery.parse("China");
say $result.perl;

69. rule query {
<city> ',' <ws>? <country>
| <city>
| <country>
}

70. rule query {
<city> ',' <ws>? <country>
| <city>
| <country>
}
SearchQuery.parse("Tirana, Albania");

71. rule query {
<city> ',' <ws>? <country>
| <city>
| <country>
}
SearchQuery.parse("Tirana, Albania");

72. Capturing
and accessing

73. Everything goes
to Match object
$/

74. SearchQuery.parse("Tirana, Albania");
say $<query><city>;
say $<query><country>;

75. SearchQuery.parse("Tirana, Albania");
say $<query><city>;
say $<query><country>;
Tirana
Albania

76. SearchQuery.parse("Tirana, Albania");
say $<query><city>; Shortcut
say $<query><country>;
say $/<query><city>; Full syntax
say $/<query><country>;

77. rule query {
'Hotels in'?
[
<city> ',' <ws>? <country>
| <city>
| <country>
]
}

78. SearchQuery.parse("Tirana, Albania");
say $<query><city>;
say $<query><country>;
SearchQuery.parse
("Hotels in Tirana, Albania");
say $<query><city>;
say $<query><country>;

79. rule date {
<day>
<month>
}
token day {
d+
['st' | 'nd' | 'th']?
}
token month {
'January'
| 'February'
| 'March'
| 'April' . . .

80. SearchQuery.parse("Hotels in Tirana,
Albania from 25th December");
SearchQuery.parse("Hotels in Tirana,
Albania from 25 December");

81. What will
$<query><date>
print?

82. What will
$<query><date>
print?
25th December
or
25 December

83. How to check days
token day {
(d+) {$0 <= 31 or fail}
}

84. [
<city> ',' <ws>? <country>
| <city>
| <country>
]
[
'from' <date>
'to' <date>
]?
[
'for' <guest_number>
]?
}

85. token guest_number {
d
| 'one'
| 'two'
| 'three'
| 'four'
| 'five'
}

86. "Hotels in Tirana, Albania from
25 December to 7 January for two"

87. rule date {
'today'
| 'tomorrow'
|[
<day>
<month>
]
}

88. $ perl6 10-all.pl
Hotels in Amsterdam, Netherlands from 1 January to 5
February for three
City: Amsterdam
Country: Netherlands
From: 1 January
To: 5 February
Guests: three

89. __END__
Andrew Shitov
talks.shitov.ru | andy@shitov.ru