Model transformations are considered to be the “heart” and “soul” of Model Driven Engineering, and as a such, advanced techniques and tools are needed for supporting the development, quality assurance, maintenance, and evolution of model transformations. Even though model transformation developers are gaining the availability of powerful languages and tools for developing, and testing model transformations, very few techniques are available to support the understanding of transformation characteristics. In this talk, a process to analyze model transformations is discussed with the aim of identifying to what extent their characteristics depend on the corresponding input and target metamodels. The process relies on a number of transformation and metamodel metrics that are calculated and properly correlated. The talk discusses the application of the approach on a corpus consisting of more than 90 ATL transformations and 70 corresponding metamodels. The slides have been used to present the paper "Mining Correlations of ATL Transformation and Metamodel Metrics" at MISE2015 workshop at ICSE2015 (http://goo.gl/UJ9nWC)

1. Dipartimento di Ingegneria e Scienze
Università degli Studi dell’Aquila
dell’Informazione e Matematica
Mining Correlations of
ATL Transformation and Metamodel
Metrics
Juri Di Rocco
Davide Di Ruscio
Ludovico Iovino
Alfonso Pierantonio

2. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
3
Introduction
Over the last decades many MDE technologies have
been conceived to support a wide range of modeling
and model management activities

3. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
4
Introduction
Availability of powerful languages and tools for
developing, testing, and chaining model
transformations
Limited support for analysing and understanding
common characteristics of model transformations
• what are the main constructs typically used when
developing transformations ?
• to what extent is the development of model
transformations affected by the complexity of the
corresponding metamodels ?

4. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
5
Introduction
Availability of powerful languages and tools for
developing, testing, and chaining model
transformations
Limited support for analysing and understanding
common characteristics of model transformations
• what are the main constructs typically used when
developing transformations ?
• to what extent is the development of model
transformations affected by the complexity of the
corresponding metamodels ?
MMs MMtT
?

5. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
6
Introduction
Several metrics are available to measure ATL
transformations
Quality attributes have been also defined and they
have been aligned to a set of metrics
None of the existing approaches deal with
transformation metrics correlation
Correlating transformation and metamodels metrics
is also unexplored

6. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
7
Contribution
Correlation of several metrics
• 28 metamodel metrics
• 35 transformation metrics
It has been applied on a corpus of
• 91 ATL transformations
• 72 corresponding metamodels
Preparatory study to estimate the required effort to develop model
transformations depending on the structural characteristics of the
input and target metamodels
Analysis process for understanding model
transformations characteristics

7. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
8
Proposed analysis process

8. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
9
Metrics calculation

9. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
10
Metrics calculation
Consists of the application of metrics on a data set of
metamodels and transformations
Sample metamodel metrics
Sample transformation metrics

10. Davide Di Ruscio
11
Metrics calculation
The metrics calculation has been implemented by
exploiting a model-driven toolchain
Artifact 1
Artifact 2
Artifact n
Metrics
Calculator
Metrics
CSV generator

11. Davide Di Ruscio
12
Metrics calculation
The metrics calculation has been implemented by
exploiting a model-driven toolchain
Artifact 1
Artifact 2
Artifact n
Metrics
Calculator
Metrics
CSV generator
The Metrics Calculator is able to
calculate for each artifact all the
considered metrics

12. Davide Di Ruscio
13
Metrics calculation
The metrics calculation has been implemented by
exploiting a model-driven toolchain
Artifact 1
Artifact 2
Artifact n
Metrics
Calculator
Metrics
CSV generator
The Metrics Calculator is an ATL
transformation whose target
models conform to the Metrics
metamodel

13. Davide Di Ruscio
14
Metrics calculation
The metrics calculation has been implemented by
exploiting a model-driven toolchain
Artifact 1
Artifact 2
Artifact n
Metrics
Calculator
Metrics
CSV generator
Generating CVS files enables the
adoption of statistical tools like
IBM SPSS, Microsoft Excel, and
Libreoffice Calc for subsequent
analysis of the generated data

14. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
15
Calculation, selection, and
statistical significance of
metrics correlation

15. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
16
Calculation of metrics correlations
Correlation is used to detect cross-links and assess
relationships among observed data
Pearson’s and Spearman’s coefficients to measure
the correlations among calculated metamodel and
transformation metrics
Pearson’s correlations Spearman’s correlations

16. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
17
Calculation of metrics correlations
Both Pearson’s and Spearman’s correlation indexes
assume values in the range of -1.00 (perfect negative
correlation) and +1.00 (perfect positive correlation)
A correlation with value 0 indicates that between two
variables there is no correlation

17. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
18
Selection of metrics correlations
Pearson’s correlation indexes for all the values of the
ATL transformation metrics
Spearman’s correlation indexes for all the values of
the ATL transformation and metamodel metrics

18. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
19
Selection of metrics correlations
ATL transformation metrics correlations
All the values greater than 0.8 have been highlighted to
select the metrics that are most related according to the
Pearson’s index.

19. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
20
Selection of metrics correlations
ATL transformation metrics correlations
All the values greater than 0.8 have been highlighted to
select the metrics that are most related according to the
Pearson’s index.
Number of output patterns (OP)
<->
number of bindings (B)

20. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
21
Selection of metrics correlations
ATL transformation metrics correlations
All the values greater than 0.8 have been highlighted to
select the metrics that are most related according to the
Pearson’s index.
Number of Transformation Rules (TR)
<->
Number of Rules with a Filter Condition on the Input Pattern (RWF)

21. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
22
Selection of metrics correlations
ATL transformation metrics correlations
All the values greater than 0.8 have been highlighted to
select the metrics that are most related according to the
Pearson’s index.
Number of Helper (H)
<->
Number of Helper with Context (HWC)

22. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
23
Selection of metrics correlations
ATL transformation and metamodel metrics
correlations
All the values greater than 0.6 have been highlighted
to select the metrics that are most related according
to the Spearman’s index.

23. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
24
Selection of metrics correlations
ATL transformation and metamodel metrics
correlations
All the values greater than 0.6 have been highlighted
to select the metrics that are most related according
to the Spearman’s index.Number of structural features in the output metamodel (SF - OUTPUT)
<->
Number of binding (B)

24. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
25
Statistical significance of
metric correlations
Just because two variables are related, it does
not necessarily mean that one directly causes
the other.
It is necessary to assess that the performed
analysis is statistically significant or not

25. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
26
Statistical significance of
metric correlations
Significance level of a statistical hypothesis refers to
the probability that the random sample that has been
chosen is not representative
• the lower the significance level, the more confident one
can be in replicating the performed results
T-test has been used to establish if the identified
correlation coefficients were statistically significant
• the threshold 0.05 has been considered
• correlations which induce a T-test value above the
threshold have been rejected

26. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
27
Data analysis
The aim is to discuss and interpret the most relevant
correlations between structural characteristics which
have been found in the previous stages

27. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
28
Data analysis
How transformation rules are influenced by target
metamodels
Number of output metaclasses
(OUT MC)
<->
Number of Transformation Rules
(TR)
Transformation development is
typically output driven (developer
tries to cover all the metaclasses
in the target metamodel)
Spearman’s index: 0.746
Significance value: 0.0002

28. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
29
Data analysis
How the total number of transformation input
patterns are influenced by the source
metamodels Number input metaclasses
(IN MC)
<->
Number of Input Patterns
(IP)
Even though not evident like in the
previous case, IN MC and IP seem
to increase together and this
might be related to the
“coverage” characteristic of the
transformations in the considered
corpus
Spearman’s index: 0.692
Significance value: 0.0001

29. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
30
Data analysis
How the structural features in the target
metamodel influence the number of bindings
Number of output
structural features
(OUT SF)
<->
Number of Bindings
(B)
Both OUT SF and B seem to
increase together and this might
be related to the “coverage”
characteristic of the
transformations in the considered
corpus
Spearman’s index: 0.808
Significance value: 0.07

30. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
31
Data analysis
How general purpose and domain specific
metamodels affect the complexity of model
transformations

31. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
32
Data analysis
How general purpose and domain specific
metamodels affect the complexity of model
transformations
Number of Rules with a Filter
Condition on the Input Pattern
In GPL2GPL and GPL2DSL transformations only parts
of metamodels and hierarchies are considered

32. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
33
Data analysis
How general purpose and domain specific
metamodels affect the complexity of model
transformations
Number of Rules with a Do Section
In GPL2GPL and DSL2GPL the use of the imperative “do” block is
higher than the other cases
Typically imperative constructs are used when the input and
output metamodels are completely different

33. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
34
Data analysis
How general purpose and domain specific
metamodels affect the complexity of model
transformations
Number of Rules with a Using clause
The use of the using clause is very limited and it seems to be one
of the less used constructs of ATL

34. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
35
Data analysis
How general purpose and domain specific
metamodels affect the complexity of model
transformations
Number of calls to resolveTemp
It seems to be never used in GPL2DSL transformations and
equally distributed in the other cases
It results to be one of the most complex construct of the
ATL language

35. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
36
Data analysis
How developers use the ATL language
Transformations in the considered corpus are mainly developed in a
declarative way:
• Most of the transformations are developed by means of matched
rules
• Helper with contexts are more than those without contexts, which
are usually used as variables in transformations described in an
imperative way

36. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
37
Conclusions
An approach to analyze model transformations by
considering also the corresponding metamodels has
been discussed
The main goal is to better understand the
characteristics of model transformations and how
their complexity is related to the complexity of
metamodels
A correlation analysis has been performed to identify
the most cross-linked metrics

37. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
38
Future work
Extend the corpus of artifacts in order to:
• validate the identified correlations
• better investigate the significance value for those
correlations that currently are below the threshold
Include in the analysis further kinds of artifacts
typically involved in any MDE approach
Rely on the results achieved so far in order to define
an approach supporting the early cost estimation for
developing model transformations

38. 7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
39
Future work
Extend the corpus of artifacts in order to:
• validate the identified correlations
• better investigate the significance value for those
correlations that currently are below the threshold
Include in the analysis further kinds of artifacts
typically involved in any MDE approach
Rely on the results achieved so far in order to define
an approach supporting the early cost estimation for
developing model transformations
Item for the panel discussion ?

39. Thank you