1. DSL Design
A conceptual framework
for building good DSLs
Markus Voelter
based on material
independent/itemis from a paper written
voelter@acm.org with Eelco Visser
www.voelter.de E.Visser@tudelft.nl
voelterblog.blogspot.de http://eelcovisser.org/
@markusvoelter
+Markus Voelter

2. Introduction

3. A DSL is a focussed, processable
language for describing a
specific concern when building a
system in a specific domain. The
abstractions and notations used
are natural/suitable for the
stakeholders who specify that
particular concern.

4. more in GPLs more in DSL
Domain Size large and complex smaller and well-defined
Designed by guru or committee a few engineers and
domain experts
Language Size large small
Turing- almost always often not
completeness
User Community large, anonymous and small, accessible and
widespread local
In-language sophisticated limited
abstraction
Lifespan years to decades months to years (driven
by context)
Evolution slow, often fast-paced
standardized
Incompatible almost impossible feasible
Changes

5. General Purpose
C
Components
Domain
State Machines Specific
Sensor Access
LEGO Robot
Control

6. Modular Language
 a b c
d e f
my
L g h i
j k l

with many optional,
composable modules

7. Case Studies

8. Example
Compo
nents

9. Example
Compo
nents

10. Example
Refrigerators
Refrige
rators

11. Example
Refrige
rators

12. Example
Refrige
rators

13. Example
Exten
ded C

14. Example
Pension
Plans

15. Example
Pension
Plans

16. Example
Web
DSL

17. Terms
&
Concepts

18. Model
A schematic description of a
system, theory, or
phenomenon that accounts for
its known or inferred
properties and may be used
for further study of its
characteristics
www.answers.com/topic/mode
l

19. Model
A representation of a set of
components of a process,
system, or subject area,
generally developed for
understanding, analysis,
improvement, and/or
replacement of the process
www.ichnet.org/glossary.htm

20. Model
which ones?
an abstraction or
simplification of
reality
what should
we leave out? ecosurvey.gmu.edu/glossary.ht
m

21. Model
Purpose
… code generation
… analysis and checking
… platform independence
… stakeholder integration
…
drives design of
language!

22. Model
Purpose
… code generation
… analysis and checking
… platform independence
… stakeholder integration
Example
Compo
nents

23. Model
Purpose
… code generation
… analysis and checking
… platform independence
… stakeholder integration
Example
Refrigerators Refrige
rators

24. Model
Purpose
… code generation
… analysis and checking
… platform independence
… stakeholder integration
Example
Extended C Exten
ded C

25. Model
Purpose
… code generation
… analysis and checking
… platform independence
… stakeholder integration
Example
Pension
Plans

26. body of
knowledge
deductive
top down in the real
world
Domain
inductive
existing bottom up
software
(family)

27. body of
knowledge
deductive
top down in the real
world
Domain
Example Example
Refrigerators Penion
Plans
Refrige
rators

28. Domain
Domain
inductive
existing bottom up
software Example Example
(family) Exten Compo
ded C netns

29. A DSL LD for D is a
language that is
specialized to en-
coding PD programs.
more efficient
smaller

30. Programs
are trees
elemen
t

31. Fragments are
subtrees w/ root
fragment root
f
fragment

32. Parent-Child
Relation
f

33. Programs
and Fragments
f

34. Programs are
graphs, really.
reference

35. Programs are
graphs, really.

36. Languages are
sets of concepts
C2
C1 Cn
C3
L

37. Languages are
sets of concepts
C2
C1 Cn
C3
L

38. Programs
and languages
C2
C1 Cn
C3

39. Language:
concept inheritance
C2
L C1
1
L2 C3
C1

40. Language
does not depend on
any other language
Independence
Fragment
does not depend on
any other fragment

41. Independence
Example
Refrige
rators

42. Homogeneous
Fragment
everything expressed
with one language

43. Heterogeneous
Heterogeneous
C
Statemachines
Testing
Example
Exten
ded C

44. Domain Hierarchy

45. Domain Hierarchy
avionics
automotive
embedded
software
Example
all programs
Exten
ded C

46. Design
Dimensions
expressivity completeness
coverage paradigms
semantics modularity
separation of concrete
concerns syntax
process

47. Expressivity
expressivity completeness
coverage paradigms
semantics modularity
separation of concrete
concerns syntax
process

48. Shorter
Programs
More
Accessible
Semantics

49. For a limited
Domain!
Domain
Knowledge
encapsulated in
language

50. Smaller Domain
More Specialized
Language
Shorter
Programs

51. The
do-what-I-
want language
Ѱ

52. Single Program
vs. Class/Domain
No Variability!
Ѱ

53. Domain Hierarchy
more specialized domains
more specialized languages

54. Reification
Dn+1
==
Dn

55. Reification
Transformation
/
Generation
==
Language
Definition

57. Overspecification!
Requires Semantic Analysis!

58. Linguistic
Abstraction
Declarative!
Directly represents Semantics.

59. Def: DSL
A DSL is a language at D that
provides linguistic abstractions
for common patterns and idioms
of a language at D-1 when used
within the domain D.

60. Def: DSL cont’d
A good DSL does not require the
use of patterns and idioms to
express semantically interesting
concepts in D.
Processing tools do not have to
do “semantic recovery” on D
programs.
Declarative!

61. Another Example
Example
Exten
ded C

62. Another Example
Example
Exten
Turing Complete!
Requires Semantic Analysis! ded C

63. Linguistic
Abstraction
Example
Exten
ded C

64. Linguistic
Abstraction
Example
Exten
ded C

65. Linguistic
Abstraction
In-Language
Abstraction
Libraries
Classes
Frameworks

66. Linguistic
Abstraction
Analyzable
Better IDE Support
In-Language
Abstraction
User-Definable
Simpler Language

67. Linguistic
Abstraction Special
Analyzable Treatment!
Better IDE Support
In-Language
Abstraction
User-Definable
Simpler Language

68. Linguistic
Abstraction
Std Lib
In-Language
Abstraction

69. Std Lib
Example
Refrige
rators

71. Coverage
expressivity completeness
coverage paradigms
semantics modularity
separation of concrete
concerns syntax
process

72. Domain DL defined
inductively by L
(the domain that can be expressed by L)
CL(L) == 1 (by definition)
not very interesting!

73. Def: Coverage
to what extend can a language
L cover a domain D

74. Def: Coverage
why would CD(L) be != 1?
1) L is deficient
2) L is intended to cover
only a subset of D,
corner cases may make L
too complex
Rest must be expressed in D-1

75. Def: Coverage
Coverage is full.
You call always write C.
Example
Exten
ded C

76. Def: Coverage
Only a particular style of
web apps are suported.
Many more are conceivable.
Example
WebDSL

77. Def: Coverage
DSLs are continuously evolved
so the relevant parts of the
deductive domain are
supported.
Example Example Example
Compo Refrige Pension
nents rators Plans

78. Semantics &
Execution
expressivity completeness
coverage paradigms
semantics modularity
separation of concrete
concerns syntax
process

79. Static Semantics
Execution Semantics

80. Static Semantics
Execution Semantics

81. Static Semantics
Constraints
Type Systems

82. Unique State Names
Unreachable States
Dead End States
…
Example
Exten
ded C

83. Unique State Names
Unreachable States
Dead End States
…
Easier to do on a
declarative Level!
Example
Exten
ded C

84. Unique State Names
Unreachable States
Dead End States
…
Easier to do on a
declarative Level!
Thinking of all
Example
constraints is a
Exten
coverage problem!
ded C

85. Assign fixed types
Derive Types
Calculate Common
Types
Check Type Consistency
What does a type system do?

86. Intent + Derive
Check
More code More convenient
Better error More complex
messages checkers
Better Performance Harder to
understand for
users

87. Example
Refrige
rators

88. What does it
all mean?
Execution Semantics

89. Def: Semantics
… via mapping to lower level
OB: Observable Behaviour (Test Cases)

90. Def: Semantics
… via mapping to lower level
LD
Transformation
Interpretation
LD-1

91. Transformation
Dn+1
Dn

92. Transformation
Example
Exten
ded C

93. Transformation
LD
Transformation
LD-1 Known
Semantics!

94. Transformation
LD
Correct!? Transformation
LD-1

95. Transformation
Tests (D) LD
Transformation
Tests (D-1) LD-1
Run tests on both levels; all pass.
Coverage Problem!

96. Example
Refrige
rators

97. Transformation
Example
Pension
Plans

98. Behavior
Example
Refrige
rators

99. Multi-Stage
L3
L2
Modularization
L1
L0

100. Multi-Stage: Reuse
L3
L2 L5
L1 Reusing
Later Stages
L0 Optimizations!

101. Multi-Stage: Reuse
L3 Robot Control
State Machine
L2 L5 Components
L1 C (MPS tree)
Example
L0 C Text Exten
ded C

102. Multi-Stage: Reuse
L3 Robot Control
State Machine
Consistency
Model Checking
L2 L5 Components
Efficient Mappings
C Type System
L1 C (MPS tree)
Syntactic
Correctness, Example
Headers
L0 C Text Exten
ded C

103. Multi-Stage: Reuse
L3 Reusing
Early Stages
L2 Portability
L1 L1b
L0 L0b

104. Multi-Stage: Reuse
L3
L2
L1 L1b Java
C#
Example
L0 L0b Exten
ded C

105. Multi-Stage:
Preprocess
Adding an
optional, modular
emergency
stop feature

106. Platform

107. Platform
Temporal
Data
Versioning
Database Access
Transactions
Distribution
(JEE)
Example
Pension
Plans

108. Platform
Data Collection
HAL
Device
Drivers
Example
Refrige
rators

109. Interpretation
A program at D0 that
acts on the structure
of an input program at D>0

110. Interpretation
A program at D0 that
acts on the structure
of an input program at D>0
imperative  step through
functional  eval recursively
declarative ? solver ?

111. Example
Pension
Plans

112. Example
Refrige
rators

113. Behavior
Example
Refrige
rators

114. Interpretation
A program at D0 that
acts on the structure
of an input program at D>0

115. Interpretation
An interpreter :-)

116. Transformation Interpretation
+ Code Inspection + Turnaround Time
+ Debugging + Runtime Change
+ Performance &
Optimization
+ Platform Con-
formance

117. Def: Semantics
… via mapping to lower level
LD
Transformation
Interpretation
LD-1

118. Multiple Mappings
… at the same time
LD
Similar
Semantics?
Lx Ly Lz

119. Multiple Mappings
… at the same time
LD T
Similar
Semantics?
Lx Ly Lz
T T T
all green!

120. Example
Pension
Plans

121. Example
Refrige
rators

122. Multiple Mappings
… alternatively, selectably
Extend LD to include
LD explicit data that
determines
transformation
Lx Ly Lz

123. Multiple Mappings
… alternatively, selectably
Extend LD to include
LD explicit data that
determines
transformation
Lx Ly Lz Example
Restricted Port leads
to reduced overhead Exten
C ded C

124. Multiple Mappings
… alternatively, selectably
External Data:
LD - Switches
- Annotation Model
Lx Ly Lz

125. Multiple Mappings
… alternatively, selectably
External Data:
LD - Switches
- Annotation Model
Switch
Control Java
vs. C
Lx Ly Lz Example
Pension
Plans

126. Multiple Mappings
… alternatively, selectably
Heuristics: Analyze
LD model to try to
decide
Lx Ly Lz

127. Multiple Mappings
… alternatively, selectably
T LD TESTING!
Lx Ly Lz
T T T

128. Reduced Expressiveness
bad? maybe.
good? maybe!
Model Checking
SAT Solving
Exhaustive Search, Proof!

129. Unique State Names
Unreachable States
Dead End States
Guard Decidability
Reachability
Exhaustive Search, Proof!

130. Example
Exten
ded C

131. Example
Exten
ded C

132. Separation of
Concerns
expressivity completeness
coverage paradigms
semantics modularity
separation of concrete
concerns syntax
process

133. Several Concerns
… in one domain

134. Several Concerns
… in one domain
integrated into separated into
one fragment several fragments

135. Viewpoints
independent
dependent

136. Viewpoints
Hardware
Behavior
Tests Example
Refrige
rators

137. Viewpoints: Why?
Sufficiency
Different Stakeholders
Different Steps in Proces
– VCS unit!

138. Viewpoints
Hardware
Product
Management Behaviour
Thermo-
dynamics-
Experts
Tests Example
Refrige
rators

139. Viewpoints
independent
sufficient?
contains all
the data for running
a meaningful
transformation

140. Viewpoints
sufficient
implementation
code
sufficient
Hardware structure
documentation
Example
Refrige
rators

141. Viewpoints: Why?
1:n Relationships

142. Viewpoints
1:n
Hardware
Behaviour
Tests Example
Refrige
rators

143. Viewpoints
Well-defined
Dependencies
No Cycles!
Avoid Synchronization!
(unless you use a projectional editor)

144. Viewpoints: Why?
LD
LAnnotation
LD-1

145. Progressive Refinement

146. Views on Programs
Example
Pension
Plans

147. Completeness
expressivity completeness
coverage paradigms
semantics modularity
separation of concrete
concerns syntax
process

148. Can you generate 100% of
the code from the DSL
program?
More generally: all of D-1

149. Semantics:
Introduce G (“generator”):
complete
incomplete

150. Incomplete: What to do?
FD
OB(FD)
!=
FD-1

151. Incomplete: What to do?
Manually written code!
FD
OB(FD)
==
FD-1 + FD-1, man

152. Manually written code?
Call “black box” code
(foreign functions)
Embed LD-1 code in LD program

153. Manually written code?
Call “black box” code
(foreign functions)
Embed LD-1 code in LD program
Embed C statements
in components, state machines
or decision tables Example
Refrige
rators

154. Manually written code?
Call “black box” code
(foreign functions)
Embed LD-1 code in LD program
Use composition mechanisms of
LD-1 (inheritance, patterns, aspects, …)

155. Manually written code?
Call “black box” code
(foreign functions)
Embed LD-1 code in LD program
Use composition mechanisms of
LD-1 (inheritance, patterns, aspects, …)
Generate base classes Example
with abstract methods; Compo
implement in subclass nents

156. Manually written code?
Call “black box” code
(foreign functions)
Embed LD-1 code in LD program
Use composition mechanisms of
LD-1 (inheritance, patterns, aspects, …)
Use protected regions (if you
really have to…)

157. Manually written code?
Call “black box” code
(foreign functions)
Embed LD-1 code in LD program
Use composition mechanisms of
LD-1 (inheritance, patterns, aspects, …)
Use protected regions (if you
really have to…) DON’T!

158. Incomplete: When?
Good for technical Example Example
DSLs: Devs write Exten Compo
LD-1 code ded C nents
Bad for business DSLs.
Maybe use a LD-1 std lib that LD
code can call into?
Example Example Example
Refrige Pension Web
rators Plans DSL

159. Prevent Breaking Promises!
class B extens BBase {
public void doSomething() {
Registry.get(„A“).someMethod();
}
}

160. Prevent Breaking Promises!
Better:
Dependency Injection
Example
Static analysis tools Compo
nents

161. Roundtripping
LD L’D
LD-1 ……… D-1
L’

162. Roundtripping – Don’t!
LD L’D
Semantic
Recovery!
LD-1 ……… D-1
L’

163. Fundamental
Paradigms
expressivity completeness
coverage paradigms
semantics modularity
separation of concrete
concerns syntax
process

164. Structure
Modularization, Visibility
Namespaces,
public/private
importing

165. Structure
Modularization, Visibility
Namespaces,
public/private
importing
divide & conquer
reuse
stakeholder integration

166. Structure
Partitioning (Files)
VCS Unit
Unit of sharing
Unit of IP
!= logical modules
may influence language design

167. Structure
Modularization, Visibility
Example
Exten
ded C

168. Structure
Modularization, Visibility
Example
Compo
nents

169. Structure
Partitioning (Files)
change impact
link storage
model organization
tool integration

170. Structure
Spec vs. Implementation
plug in different Impls
different stakeholders

171. Structure
Spec vs. Impl.
Example
Exten
ded C

172. Structure
Specialization
Liskov substitution P
leaving holes (“abstract”)

173. Structure
Specialization
Liskov substitution P
leaving holes (“abstract”)
variants (in space)
evolution (over time)

174. Structure
Specialization
Pension Plans can inherit
from other plans.
Rules can be abstract;
Plans with abstract Example
rules are abstact
Pension
Plans

175. Structure
Superposition, Aspects
merging
overlay
AOP
modularize cross-cuts

176. Structure
Superposition, Aspects
Example
Compo
nents

177. Behavior
Not all DSLs specify behavior
Some just declare behavior
This section is
not for those!

178. Behavior
Imperative
sequence of statements
changes program state
write understand debug analyze performance
simple simple - simpl hard good
e

179. Behavior
Imperative
sequence of statements
changes program state
Example
Refrige
rators

180. Behavior
Functional
functions call other functions.
no state. No aliasing.
write understand debug analyze performance
simple - simple simpl good good -
e

181. Behavior
Functional
functions call other functions.
no state. No aliasing.
Example
Pension
Plans

182. Behavior
Functional
Example
Pension
Plans

183. Behavior
Functional
pure expressions are
a subset of functional
(operators hard-wired)
guards
preconditions
derived attributes

184. Behavior
Declarative
only facts and goals.
no control flow.
eval engine, solver (several)
write understand debug analyze performance
simple simple - hard depends often bad

185. Behavior
Declarative
concurrency
constraint programming
solving
logic programming

186. Behavior
Declarative
only facts and goals.
no control flow.
eval engine, solver (several)
Example
Web
DSL

187. Behavior
Declarative
Example
Exten
ded C

188. Behavior
Reactive
reactions to events, more
events are produced.
Communication via
events and channels/queues
write understand debu analyze performance
g
simple simple/har hard simple can be good

189. Behavior Reactive
Example
Refrige
rators

190. Behavior
Data Flow
chained blocks consume
continuous data that flows
from block to block
write understand debu analyze performance
g
simple - simple/har hard simple can be good

191. Behavior
Data Flow
continuous, calc on change
quantized, calc on new data
time triggered, calc every x

192. Behavior
Data Flow
Embedded Programming
Enterprise ETL & CEP

193. Behavior
State Based
states, transitions,
guards, reactions
event driven, timed
write understand debu analyze performance
g
simple - simple/har s/h simple + can be good

194. Behavior
State Based
Example
Refrige
rators

195. Behavior
Combinations
data flow uses functional,
imperative or declarative lang
inside block

196. Behavior
Combinations
state machines use expressions
in guards and often an
imperative lang in actions

197. Behavior
Combinations
Example
Refrige
rators

198. Behavior

199. Behavior
Combinations
purely structural languages
often use expressions to specify
constraints
Example
Exten
ded C

200. Language
Modularity
expressivity completeness
coverage paradigms
semantics modularity
separation of concrete
concerns syntax
process

201. Language Modularity,
Behavior and Reuse (LMR&C)
Composition
increase efficiency
of DSL development

202. Language Modularity,
Behavior and Reuse (LMR&C)
Composition
increase efficiency
of DSL development
4 ways of composition:
Referencing
Reuse
Extension
Reuse

203. Language Modularity,
Behavior and Reuse (LMR&C)
Composition
increase efficiency
of DSL development
4 ways of composition:
distinguished regarding
dependencies and fragment
structure

204. Dependencies:
Behavior
do we have to know about the
reuse when designing the
languages?

205. Dependencies:
Behavior
do we have to know about the
reuse when designing the
languages?
Fragment Structure:
homogeneous vs. heterogeneous
(„mixing languages“)

206. Dependencies &
Behavior
Fragment Structure:

207. Dependencies &
Behavior
Fragment Structure:

208. Referencing
Referencing

209. Referencing
Referencing
Dependen
t
No containment

210. Referencing
Used in
Viewpoints

211. Referencing
Fragment
Fragment
Fragment

212. Referencing
Example
Refrige
rators

213. Extension
Dependen
t
Containment

214. Extension
Extension
more specialized domains
more specialized languages

215. Extension
Dn+1
==
Dn

216. Extension
Dn+1
==
Dn

217. Extension
Good for bottom-up (inductive)
domains, and for use by
technical DSLs (people)
==
Dn

218. Extension
Drawbacks
Behavior
tightly bound to base
potentially hard to analyze
the combined program

219. Extension
Example
Exten
ded C

220. Extension
Extension
Example
Exten
ded C

221. Reuse
Reuse
Independent
No containment

222. Reuse
Reuse
Behavior referenced language
Often the
is built expecting it will be
reused.
Hooks may be added.

223. Embedding
Independent
Containment

224. Embedding
Example
Pension
Plans

225. Embedding
Behavior
Embedding often uses Extension
to extend the embedded
language to adapt it to its new
context.

226. Challenges - Syntax
Behavior and Embedding
Extension
requires modular concrete
syntax
Many tools/formalisms cannot
do that

227. Challenges – Type Systems
Behavior the type system of
Extension:
the base language must be
designed to be extensible/
overridable

228. Challenges – Type Systems
Behavior
Reuse and Embedding: Rules that
affect the interplay can reside
in the adapter language.

229. Challenges – Trafo & Gen
Referencing (I)
Behavior
Written
specifically Can be
for the Reused
combination
Two separate, dependent
single-source transformations

230. Challenges – Trafo & Gen
Referencing (II)
A single multi-sourced
transformation

231. Challenges – Trafo & Gen
Referencing (III)
A preprocessing trafo that changes the
referenced frag in a way specified by
the referencing frag

232. Challenges – Trafo & Gen
Extension
Transformation by assimiliation, i.e.
generating code in the host lang
from code expr in the extension lang.

233. Challenges – Trafo & Gen
Extension
Example
Exten
ded C

234. Challenges – Trafo & Gen
Reuse (I)
Reuse of existing
transformations for both fragments
plus generation of adapter code

235. Challenges – Trafo & Gen
Reuse (II)
composing transformations

236. Challenges – Trafo & Gen
Reuse (III)
generating separate artifacts
plus a weaving specification

237. Challenges – Trafo & Gen
Embedding (I)
a purely embeddable language
may not come with a generator.
Assimilation (as with Extension)

238. Challenges – Trafo & Gen
Embedding (II)
Adapter language can coordinate the
transformations for the host and for
the emebedded languages.

239. Concrete
Syntax
expressivity completeness
coverage paradigms
semantics modularity
separation of concrete
concerns syntax
process

240. UI for the language!
Important for acceptance by users!
Textual
Symbolic
Tabular
Graphical

241. Reuse existing
syntax of
domain, if any!
Tools let you
freely combine
all kinds.

242. Default: Text
Editors simple to build
Productive
Easy to integrate w/ tools
Easy to evolve programs

243. Default: Text
Editors simple to build
Productive
Easy to integrate w/ tools
Easy to evolve programs
… then add other forms,
if really necessary

244. Graphical in case…
Relationships

245. Graphical in case…
Flow and
Depenency

246. Graphical in case…
Causality
and Timing

247. Symbolic
Either
Mathematical,
or often
highly
inspired by
domain

248. Tables

249. Combinations

250. Combinations

251. Combinations

252. Combinations

253. Process
expressivity completeness
coverage paradigms
semantics modularity
separation of concrete
concerns syntax
process

254. Domain Analysis
Interview
Experts
Get Structure
their
Feedback Knowledge
Create Build the
Examples Language

255. Iterate to goal

256. Documentation
Create example-
based tutorials!

257. Domain Folks
Programming?
Precision vs.
Algorithmics!

258. Domain Folks
Programming?
DU
Coding
DU/Dev
Paired
Dev Coding
DU Reviewing

259. DSL as a Product
Release Plan
Bug Tracker
Testing!
Support
Documentation

260. Reviews
Reviews become
easier --- less
code, more
domain-specific

261. The End.
This material is part of my
upcoming (early 2013) book
DSL Engineering with
Language Workbenches
Stay in touch; it will be cheap or
maybe even free :-)
www.voelter.de/dslbook
@markusvoelter
+Markus Voelter