Talk about DSL, How to write DSL in Clojure, How to use Instaparse (simplest library for parsing grammars) and how we use Clojure and Instaparse in Zoomdata

1. DSL in Clojure
Misha Kozik
@mishadoff

2. developers domain
experts

3. developer domain
expert
Badass Guys
developers domain
experts

4. developer domain
expert
Badass Guys
developers domain
experts
“DSL-powered”
Badass Guys

5. not Turing Complete

6. SQL
select statement
DSL

7. SQL
DSL
query plan

8. SQL
algorithms
DSL

9. SQL
DSL
nested loop join
#1
#2
#1
Outer Inner
#3
#1
#2
#1
#3
#4

10. SQL
DSL
nested loop join
#1
#2
#1
Outer Inner
#3
#1
#2
#1
#3
#4

11. SQL
DSL
hash join
#1
#2
#1
Outer Inner (hashed)
#3
#1 #2 #3
#2
#1 #4
cd632f
0980eb
48b5d4

12. SQL
DSL
hash join
#1
#2
#1
Outer Inner (hashed)
#3
#1 #2 #3
#2
#1 #4
cd632f
0980eb
48b5d4

13. SQL
DSL
merge join
#106
#123
Outer (sorted) Inner (sorted)
#133
#106
#106
#117
#123

14. SQL
DSL
merge join
#106
#123
Outer (sorted) Inner (sorted)
#133
#106
#106
#117
#123

15. SQL
DSL
it’s just a join

16. SQL-on-Everything
LINGUAL
CASCADING

17. Hiccup
DSL

18. Hiccup
DSL

19. Hiccup
DSL

20. Hiccup
DSL

21. Lein
DSL

22. Binary Tree

23. Binary Tree

24. Internal DSL: Tree
Clojure way
head
left child right child

25. Internal DSL: Tree
Java way

26. External DSL: Tree
wow, such declarative
Do
not read
this code!
Domain Expert way

27. Generative DSL: Macros
excerpt macro from core.async

28. Instaparse
What if context-free grammars were
as easy to use as regular expressions?

29. Context-Free Grammar

30. Context-Free Grammar

31. Context-Free Grammar

32. Context-Free Grammar

33. Context-Free Grammar

34. Balanced Parens
Example

35. Balanced Parens
Example

36. Clojure is a functional programming language better than java
English Grammar
Example

37. English Grammar
Example
Named Entity Recognition
Kiev is the capital of Ukraine
Entity:City Entity:Country

38. English Grammar
Example
Sentiment Analysis

39. SQL-2003 Grammar
Example
1300+ rules
http://savage.net.au/SQL/sql-2003-2.bnf

40. Integration Story

42. Grouping Metric

43. Grouping Metric

44. Grouping Metric
DSL to define formula

48. JVM
Why Clojure?

49. Lisp
Why Clojure?

50. Code is Data
Why Clojure?

51. Immutable
Why Clojure?

52. REPL
Why Clojure?

53. Practical
Why Clojure?

54. Community
Why Clojure?

55. Parser Development

56. insta/transform
Development

57. insta/transform
Development

58. Syntax Error
Exception Handling

59. Tree Tests
Testing
Expected
Actual

60. Starting Rule
Testing
Isolated Parsing

61. Performance Tests
Testing
clojure.test does not support timeouts, write by yourself

62. Ambiguity Tests
Testing
instaparse is able to handle ambiguity grammars
though, still good to know where we behave poorly

63. Use Cases Tests
Testing
Year over Year growth

64. Interface
Java Integration

65. clojure-maven-plugin
Java Integration

66. Demo Time

67. ANTLR
Alternatives
http://stackoverflow.com/questions/1931307/antlr-is-there-a-simple-example

68. parboiled
Alternatives

69. Lessons
Learned

70. Lessons Learned #1
Too Much Rules100+ rules, possible to reduce to 50

71. Lessons Learned #2
Lein and Maven
are not friends

72. Lessons Learned #3
Code Coverage
not integrated

73. Lessons Learned #4
Versioning
as a library

74. Lessons Learned #5
Evaluation
is not a parser responsibility

75. Questions?