Slides for the presentation at EclipseCon Europe 2013. For more details, see http://www.eclipsecon.org/europe2013/xcore-meets-incquery-how-new-generation-dsls-are-made http://incquery.net/blog/2013/10/xcore-meets-incquery-how-new-generation-dsls-are-made-talk-eclipsecon-europe-2013

1. XCORE MEETS INCQUERY:
HOW THE NEW GENERATION
OF DSLS ARE MADE
Tamás Szabó (itemis AG), István Ráth (BUTE)
With contributions from:
Ed Merks
Gábor Bergmann, Ábel Hegedüs, Ákos Horváth,
András Ökrös, Zoltán Ujhelyi, Benedek Izsó
Budapest University of Technology and Economics
Fault Tolerant Systems Research Group
IncQueryLabs Ltd.
Itemis AG

2. Overview




Introduction
Live graph search for EMF
Query-based language engineering
Conclusion

3. EMF-INCQUERY:
LIVE GRAPH SEARCH FOR EMF

4. Queries in modeling tools
 Important!
o Model management
o Queries
o Views
o Transformations

5. Incremental queries
 Key observation: model queries are expensive
o Models are large (1-5-10+ M elements)
o Queries are complex (attribute value checks +
navigation/joins + iterations)
o Industrial example: well-formedness validation of a 4M
element AUTOSAR model takes 30+ minutes
• Despite being hand-coded in Java (no OCL, …)
 Usability issues
 Productivity issues
 How to address this?
o Lazy evaluation
o Incrementality: calculate once, update afterwards

6. Comparison
 Normal queries
 Live (incremental) queries
o Java, OCL, …
Quer
y
Model
Evaluator
Results
Model change
(Re-)traversal necessary if
the model is changed
Model change
Quer
y
Model
Evaluator
Updated results
Efficient change
propagation
Result deltas
1. Always up-to-date results
without model traversal
2. Track changes of your model
in terms of queries

7. EMF-INCQUERY
 An EMF model query engine for live graph search
• Expressive query
language based on
graph patterns and
Xbase
• Incremental
evaluation based on
the Rete algorithm
• Instantaneous
response for
complex queries
over large models
• Incremental model
processing (validation,
synchronization, ...)
Language
Execution
Tools
http://eclipse.org/incquery

8. Important use-case: Live abstractions
abstract
Complex model
Computed overlay
aka. “View”
Defined by a query
Items = SELECT …
Id
Label
Prop0
Prop1
0
N1
a
B
1
N2
c
D

9. Important use-case: Live abstractions
UI update
Model
Modification
abstract
Complex model
Computed overlay
aka. “View”
Change notification
Defined by a query
Items = SELECT …
Id
Prop0
Prop1
0
N1
a
B
1
Query result update
Label
N2
c
D
2
N3
e
F

10. INCQUERY Viewers
1. Model
Modification
On-the-fly
abstractions over
the model
EMF Model
2. Change
Notifications
Live
Live
Queries
Queries
Labeled, hierarchic
property graph
Derived
Model
3. Continuous,
efficient
synchronization
UI
4. UI updates
 Visualize things that are not (directly) present in your model
 Provides an easy-to-use API for integration into your presentation layer
o Eclipse Data Binding
o Simple callbacks

11. DEMO INCQUERY Viewers
 Using pattern annotations for the specification of
on-thy-fly abstractions
 Using the INCQUERY Viewers Sandbox for
development and testing
 Visualizing large graphs with yFiles
 Using INCQUERY Viewers Extensions APIs in your
own apps

12. Live abstractions: simple example
 Influence relationships in the Library
W1
W1: Writer
W2: Writer
: writers
B1: Book
: writers
: cites
B2: Book
: influencedBy
Derived relationship
W2
: citations
: citations
Bx: Book
Transitive closure
EMF instance model
Visualization

13. QUERY-BASED LANGUAGE
ENGINEERING

14. Xcore
 Xcore
o Extended textual syntax for
Ecore-based
metamodeling, built on Xtext
o Fully fledged programming
language
o To specify structure +
behavior
(operations, datatype
conversion logic)
o http://wiki.eclipse.org/Xcore
Xbase
• Xtext-based programming
language
• Compiles to Java
• Java type system
• Supports advanced constructs
(closures, type inference,
operator overloading, …)
• http://wiki.eclipse.org/Xbase

15. Derived features in Ecore
 Getters (and setters) implemented with Java
 Frequently used to transparently integrate
complex queries into the language
o Can also be specified by means other than Java
 Nice proposition
o Transparent: integrate queries into the model – no
utility classes needed
o Extend the expressive power of metamodeling
 Many language engineers are reluctant to use
them – why?

16. The functionality challenge
 Typical modeling tool
architecture
View
o Model: EMF
o Controller: ~ EMF.Edit
o View: e.g. JFace Viewers
 Change notifications are
essential for this to work
 EMF has nice notifications
o Notifier - Adapter
o … except for derived
features
Controller
Change notifications
Model

17. The performance challenge
 What happens when a getter is called?
class Book {
String title = "" // set a default value
refers Writer[] writers
derived String longName get {
return title + " (" + writers.size + ")"
}
derived String shortName get {
return longName.subString(0,5)
}
}
dependencies
Tougher problem than you
 Dependencies
might think:
o The value of “longName” depends on “title” and “writers”
- DFs are used explicit
o The value of “shortName” depends on “longName” –a lot in dependency
complex
• … and thus on “title” and “writers” – implicit dependency DSLs
- Query complexity times
 Performance challenge: getters may be called manycan
explode easily
o Make sure they execute quickly – but how?
o Cache whenever you can – but how?

18. Solution: well-behaving derived features
 Key idea
o Changes in the value of “name” depend on changes in
the values of “title” and “writers”
o Change notifications can be used to implement an
efficient caching strategy
o As long as every feature is behaving well
• i.e. sends notifications when necessary
 But…
o How do I discover implicit dependencies? graph
Live
search!
o How do I get/send notifications for derived features?

19. DEMO Query integration options for INCQUERY
 INCQUERY Java API
 Add-on APIs
o INCQUERY Validation
o INCQUERY Viewers

20. DEMO Query integration options for INCQUERY
 INCQUERY Java API
 Add-on APIs
o INCQUERY Validation
o INCQUERY Viewers
 Query-based derived features
o Automatically computes and maintains references
and attribute values
o Transparent integration with Xcore and (pure) Xbase
• Works with traditional genmodeled Ecore too
o Unique feature: well-behaving derived features with
efficient and automated notification support

21. Scalability? Yes!
Eclipse OCL, SPARQL
engines, Neo4j, …
Characteristic
difference, note
the log scales
• IncQuery can provide near zero response times for
complex queries over models up to 14M elements
•
Check http://incquery.net/publications/trainbenchmark for
more details

22. CONCLUSION

23. What can I do with all this? – query-based live abstractions
Eclipse
technology
Pros
Trees, tables,
Properties
(JFace viewers)
EMF.Edit
The real deal:
doesn’t hide abstract syntax
Diagrams
GEF, GMF,
Graphiti
Easy to read and write
for non-programmers
Textual DSLs
Xtext
Easy to read and write
for programmers
INCQUERY
Viewers
Makes understanding and
working with complex models
a lot easier
Syntax
JFace, Zest,
yFiles
Your tool!

24. What can I do with all this? – query-based derived features
 Boost the power and scalability of Ecore:
live graph search tightly integrated into your DSL
o Self validating models
• 30+ min batch  <1s on-the-fly
o Query-based traceability links / language integration
o INCQUERY Viewers  “derived EObjects”
o << your idea here >>
 Get rid of lots of legacy stuff
o Legacy model elements
o Legacy helper/utility classes
o Legacy UI boilerplate

25. Top 3 cool applications of INCQUERY
Model-driven development toolchain for avionics systems,
Based on a combination of Eclipse tech and MATLAB/Simulink
Well-formedness validation in a modeling tool for an automotive OEM
Development tools for live dependency analysis of a large software infrastructure model

26. Three points to take home
1. Live abstractions are very powerful.
2. Use well-behaving derived features to boost your
DSL’s expressive power.
3. Xcore+INCQUERY is how the new generation of
DSLs are made 

27. Final points
 Find all details, examples, demos at
o http://eclipse.org/incquery
o http://incquery.net
 Check out the blog post
o http://incquery.net/blog/2013/10/xcore-meets-incqueryhow-new-generation-dsls-are-made-talk-eclipseconeurope-2013
o Demo projects
o Further reading (docs + the theory behind)
 Release 0.8 (including all these new features) is
scheduled shortly after the Xtext 2.5 release
 Your contributions (feedback, forum posts, ideas,
bugzillas, patches) are very welcome!