Slides from the talk at ScalaBay meetup in Oct 2015

1. Building DSLs with
Scala
Mohit Jaggi
Code Ninja and Big (Data) Troublemaker
Ayasdi

2. Who am I?
• Software engineer and architect
• Past life: built networking, security and
application delivery appliances using ASICs,
microcode, C, some C++. Operating system
optional.
• This life: distributed systems, (big) data analytics,
machine learning using Java and Scala. And real
servers with a proper operating system!

3. Why I love Scala?
• Less red tape
• Multi-paradigm
• Exploits the Java ecosystem
• Static typing and type inference
• Same language for everybody from casual script
writer to application programmer to library
designer

4. Who will find this useful?
• Beginner to intermediate scala programmers
• If you write code, you have APIs
• Library designers

5. Agenda
• Motivation for DSLs
• Scala Constructs useful for DSLs
• Lessons from my limited experience

6. Motivation for DSLs
• API Design Considerations
• DSL
• Types of DSLs and tradeoffs

7. Considerations for a good
API
• sufficient for current needs
• extensible for anticipated needs
• does not preclude unanticipated needs
• forward/backward compatibility
• easy to use
• little or no documentation required, but
• complete documentation available

8. DSL
• Domain Specific Language
• helps with ease of use part
• e.g. SQL, R, jooq, bigdf

9. Types of DSLs
• External or standalone
• Internal or embedded

10. External DSL
• New language from “scratch”
• Write a parser using
something like lex/yacc, antlr
• Write an interpreter or
• Write a code generator
• Manage a run-time
environment(Java?, native?,
REPL?)

11. Internal DSL
• “internal” or “embedded”
in a general purpose
programming language
• Designed to “feel like” a
different language more
intuitive to the domain
• e.g. Jooq in Java/Scala
feels like SQL

12. Tradeoffs
• Less work
• More completeness
• Less syntax flexibility
• “global optimizations” hard
• Error messages less
meaningful
• More freedom in syntax
• More “global optimizations”
possible
• Possible to have better error
messages
• More work
• Typically slower without
good code generation
I
n
t
e
r
n
a
l
E
x
t
e
r
n
a
l
Benefit Cost

13. Why Scala for DSLs?
select(BOOK.ID) from BOOK where BOOK.TITLE === “Scala scala”
select(BOOK.ID).from(BOOK).where(BOOK.TITLE.equal(“Java java”))

14. Scala Constructs
Useful For DSLs

15. Vanishing Method Call
• apply() and update() can be called without
naming them
df(“age”) == df.apply(“age”) == df.column(“age”)
df(“age”) = df(“age”) + 1
df.update(df(“age”), df(“age”) + 1)

16. Simulated dynamic typing
• extend the Dynamic trait
• hides compile errors, is it worth paying that cost?
df.age == df(“age”) == df.selectDynamic(“age”)
def selectDynamic(colName: String) = {
val col = self.column(colName)
if (col == null) logger.error(s"$colName does not match any DF API or column
name")
col
}
def updateDynamic(colName: String)(that: Column): Unit = update(colName, that)
df.age = df.age + 1
df.updateDynamic(“age”)(age + 1)

17. Dial 0 for operator
• dot is optional, so methods looks like operators
• any symbol can be used as operator
• for example to list people that can buy beer you
can use
df.where(df(“age”).gt(21))
df where (df(“age”) > 21)

18. • operator ending in colon applies to right
operand, this provides more freedom (~active vs
passive voice)
object orange {
def eat_:(who: String):Unit =
println(s"$who is eating orange”)
}
“monkey” eat_: orange
• careful with operators like ==, use === instead

19. Free companion object
• hide “new” to make an object
val df = DF.fromCSVFile(…)
• also a good place to put implicit conversions in

20. Reading between the lines
• implicit conversion to “enrich” types
• can also use “implicit class”
• newer construct “Value Classes” can sometimes replace this
2 MB == ByteSize(2*1024*1024)
class BytesMaker(n: Int) {
def MB = new Bytes(n * 1024 * 1024)
def GB = new Bytes(n * 1024 * 1024 * 1024)
}
case class Bytes(val n: Int) { def print = println(s”bytes=$n") }
object Bytes {
import scala.language.implicitConversions
implicit def intToBytesMaker(n: Int) = new BytesMaker(n)
}

21. My name is Bond
• Pass by name parameters are useful to pass in
“blocks of code”
def doTwice(action: => Unit) {
action
action
}
doTwice { println("hey") }

22. Multiple Parameter Lists
• Help avoid unwanted commas and more intuitive
parentheses/braces
def when(predicate: => Boolean)(action: => Unit) = {
if(predicate) action
}
when(1 == 1) { println(“1”) }
instead of
when(1 == 1, println(“1”))

23. FWIW - API
• Don't provide unnecessary options, if needed
you can always add more; like "salt in a dish",
you can't remove it
• Names meaningful to user not the coder
• Consistent coding style
• Option[T] if truly optional, else exception

24. FWIW - DSL
• Don’t take it to extremes (like “Baysick”)
• Assume users know some(or lot of) Scala
• Try to generate useful error messages
• Careful with ==, right associative operators, implicit
search order
• Aim to provide gentle slope to Scala instead of none;
basic Scala is simple, your API users will appreciate it

25. Thanks!
We are hiring
http://engineering.ayasdi.com
http://www.ayasdi.com/careers