Assessing Product Line Derivation
Operators Applied to Java Source Code:
An Empirical Study
João Bosco Ferreira Filho, Sim...
2
Feature
0..1
1
0
1
1
1
Variation Points
Variability is omnipresent
in numerous kinds of artefacts
Given a kind of artefact (expressed in a language)
you want to make it vary
You want variants of a…
3
Word document
Java, ...
You need a solution for (de)activating/adding/
removing, substituting some elements; and thus
deriving variants of a…
4
Id...
Common Variability Language (CVL): automatically
deriving products (eg models)
5
0..1
0
1
1
0
1
1
t1 t3
t2
A B C
Object Ex...
6
0..1
0
1
1
0
1
1
t1 t3
t2
A B C
Object Existence1
Link Existence1
Link Existence2
Link Existence3
Derivation Engine
t1
t...
7
0..1
0
1
1
0
1
1
Object Existence1
Link Existence1
Link Existence2
Link Existence3
Derivation Engine
Derivation Operator...
e.g., automatically FIND THESE PERCENTAGES!
Which derivation operations are more likely to work? derive uncompilable code?...
e.g., automatically FIND THESE PERCENTAGES!
Which derivation operations are more likely to work? derive uncompilable code?...
10
http://grepcode.com/file/repo1.maven.org/maven2/com.codahale.metrics/metrics-core/
3.0.0-BETA3/com/codahale/metrics/Csv...
Kind of derivation operations by example
11
Object existence expresses whether
a determined object will make part or not o...
Kind of derivation operations by example
12
Object Substitution expresses that a determined program
element will be replac...
Kind of derivation operations
13
Object existence expresses whether
a determined object will make part or not of the deriv...
e.g., automatically FIND THESE PERCENTAGES!
Which derivation operations are more likely to work? derive uncompilable code?...
Empirical
Study
15
Methodology
16
p
p'
Object Existence
Link Existence
Object Subst.
LinkEnd Subst.
CVLVPs
Derivation

Operation
Compilation
...
Study (the big picture)
17
p'
86 CVL operations
Derivation
 Compilation
and Testing
Object Existence Field
Object Existenc...
Subject Programs
18
All have JUnit tests (statement coverage greater than 70%)
Variables and measurements
19
Derivation

Object Existence Field
Object Existence Interface
Object Existence Foreach
Objec...
Instrumentation
20
Instrumentation (2)
21
370,000 programs
one month of computation
Compilation
and Testing
Results and
Findings
22
Results and Findings
• 86% of the possible pairs (CVL operator + Type of Java
Code Element) resulted in compilable program...
Results and Findings
• There are operations that will always lead to
counterexamples or to variants
24
Success Rate
~100%
...
Results and Findings
• Operations with low success rate may not be directly
discarded, but specialized
▪Looking into those...
Visualization Tool
26
Project

View
Class

View
Operation

View
1 2
3
Results and Findings
• Specialization to avoid recurrent errors
▪Simple specializations
• e.g., removing a try in the case...
Results and Findings
• Varying entire blocks of code instead of single instructions is
more likely to generate correct pro...
Results and Findings
• Object Existence is more likely to generate variants
▪ Object Existence à 21.97% of variants
▪ Lin...
Conclusions
• Large-scale assessment of derivation operations
▪More than 370,000 derived products
• Many kinds of operatio...
Future Work
• Using the results to
▪Devise specialized derivation operators for Java
▪Help current variability supporting ...
32
Object Existence2
Object Existence3
Object Existence4
Object Existence5
VS1
VS6
VS4
VS5
VS2
0..2
VS3
Object Existence1
...
Prochain SlideShare
Chargement dans…5
×

Assessing Product Line Derivation Operators Applied to Java Source Code: An Empirical Study

749 vues

Publié le

Product Derivation is a key activity in Software Product Line Engineering. During this process, derivation operators modify or create core assets (e.g., model elements, source code instructions, components) by adding, removing or substituting them according to a given configuration. The result is a derived product that generally needs to conform to a programming or modeling language. Some operators lead to invalid products when applied to certain assets, some others do not; knowing this in advance can help to better use them, however this is challenging, specially if we consider assets expressed in extensive and complex languages such as Java. In this paper, we empirically answer the following question: which product line operators, applied to which program elements, can synthesize variants of programs that are incorrect, correct or perhaps even conforming to test suites? We implement source code transformations, based on the derivation operators of the Common Variability Language. We automatically synthesize more than 370,000 program variants from a set of 8 real large Java projects (up to 85,000 lines of code), obtaining an extensive panorama of the sanity of the operations.

Paper was presented at SPLC'15

Publié dans : Sciences
0 commentaire
0 j’aime
Statistiques
Remarques
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Aucun téléchargement
Vues
Nombre de vues
749
Sur SlideShare
0
Issues des intégrations
0
Intégrations
34
Actions
Partages
0
Téléchargements
2
Commentaires
0
J’aime
0
Intégrations 0
Aucune incorporation

Aucune remarque pour cette diapositive

Assessing Product Line Derivation Operators Applied to Java Source Code: An Empirical Study

  1. 1. Assessing Product Line Derivation Operators Applied to Java Source Code: An Empirical Study João Bosco Ferreira Filho, Simon Allier, 
 Olivier Barais, Mathieu Acher and Benoit Baudry 
 SPLC 2015, July 20 - 24, 2015, Nashville,TN, USA
  2. 2. 2 Feature 0..1 1 0 1 1 1 Variation Points Variability is omnipresent in numerous kinds of artefacts
  3. 3. Given a kind of artefact (expressed in a language) you want to make it vary You want variants of a… 3 Word document Java, HTML, CSS or C++ program Class diagram State machine model A B C t1 t3 t2 A B C t2 A C
  4. 4. You need a solution for (de)activating/adding/ removing, substituting some elements; and thus deriving variants of a… 4 Ideally applicable to any kind of artefact expressed in a language Word document Java, HTML, CSS or C++ program Class diagram State machine model A B C t1 t3 t2 A B C t2 A C
  5. 5. Common Variability Language (CVL): automatically deriving products (eg models) 5 0..1 0 1 1 0 1 1 t1 t3 t2 A B C Object Existence1 Link Existence1 Link Existence2 Link Existence3 Derivation Engine Derivation Operators t1 t3A B C t2 A C Derived Products Ideally applicable to any kind of artefact expressed in a language (conformant to a metamodel)
  6. 6. 6 0..1 0 1 1 0 1 1 t1 t3 t2 A B C Object Existence1 Link Existence1 Link Existence2 Link Existence3 Derivation Engine t1 t2 A C Derivation Operators t1 t3A B C t2 A C Derived Products Previous work show #1 Using CVL “as is” is not working. It is highly beneficial to specialize derivation operators for a given language [Filho et al. SPLC’13]; mandatory in industrial context [Filho et al. STTT’14] #2 Hard for users to do not make mistake: verification techniques eg [Czarnecki et al. GPCE’06, Batory et al. GPCE’07, Classen et al. ICSE’10]; or support for preventing errors and guiding users when specifying variability in an IDE
  7. 7. 7 0..1 0 1 1 0 1 1 Object Existence1 Link Existence1 Link Existence2 Link Existence3 Derivation Engine Derivation Operators Derived Products Previous work show #1 Using CVL “as is” is not working. It is highly beneficial to specialize derivation operators for a given language [Filho et al. SPLC’13]; mandatory in industrial context [Filho et al. STTT’14] #2 Hard for users to do not make mistake: verification techniques eg [Czarnecki et al. GPCE’06, Batory et al. GPCE’07, Classen et al. ICSE’10]; or support for preventing errors and guiding users when specifying variability in an IDE
  8. 8. e.g., automatically FIND THESE PERCENTAGES! Which derivation operations are more likely to work? derive uncompilable code? Can we identify operations subject to specialization? 8 #1 Using CVL “as is” is not working. We need to specialize derivation operators for a given language [Filho et al. SPLC’13] [Filho et al. STTT’14] #2 Hard for users to do not make mistake: support for preventing errors and guiding users when specifying variability in an IDE Object Existence2 Object Existence3 Object Existence4 Object Existence5 VS1 VS6 VS4 VS5 VS2 0..2 VS3 Object Existence1 >80% <10% >40% t1 t3 t2 A B C For any programming/ modeling “language”
  9. 9. e.g., automatically FIND THESE PERCENTAGES! Which derivation operations are more likely to work? derive uncompilable code? Can we identify operations subject to specialization? 9 #1 Using CVL “as is” is not working. We need to specialize derivation operators for a given language [Filho et al. SPLC’13] [Filho et al. STTT’14] #2 Hard for users to do not make mistake: support for preventing errors and guiding users when specifying variability in an IDE Object Existence2 Object Existence3 Object Existence4 Object Existence5 VS1 VS6 VS4 VS5 VS2 0..2 VS3 Object Existence1 >80% <10% >40%
  10. 10. 10 http://grepcode.com/file/repo1.maven.org/maven2/com.codahale.metrics/metrics-core/ 3.0.0-BETA3/com/codahale/metrics/CsvReporter.java Object Existence1 Object Existence2 Object Existence3
  11. 11. Kind of derivation operations by example 11 Object existence expresses whether a determined object will make part or not of the derived variant; its execution implies on deleting or adding any source code element (e.g., statements, assignments, blocks, literals, etc.) from the original program.
  12. 12. Kind of derivation operations by example 12 Object Substitution expresses that a determined program element will be replaced by another of its same type
  13. 13. Kind of derivation operations 13 Object existence expresses whether a determined object will make part or not of the derived variant; its execution implies on deleting or adding any source code element (e.g., statements, assignments, blocks, literals, etc.) from the original program. Link Existence expresses whether there is a relationship or not between two elements, in the case of Java programs, we consider as a link any relationship between classes: association, composition, inheritance, etc (e.g., to remove an extends Class A from a class' header). Object Substitution expresses that a determined program element will be replaced by another of its same type, e.g., a method substituted by another method. Link End Substitution expresses that a relationship between a class A and a class B will be replaced by another relationship of the same type between class A and a third class C (e.g., A extends C instead of A extends B).
  14. 14. e.g., automatically FIND THESE PERCENTAGES! Which derivation operations are more likely to work? derive uncompilable code? Can we identify operations subject to specialization? 14 #1 Using CVL “as is” is not working. We need to specialize derivation operators for a given language [Filho et al. SPLC’13] [Filho et al. STTT’14] #2 Hard for users to do not make mistake: support for preventing errors and guiding users when specifying variability in an IDE Object Existence2 Object Existence3 Object Existence4 Object Existence5 VS1 VS6 VS4 VS5 VS2 0..2 VS3 Object Existence1 >80% <10% >40%
  15. 15. Empirical Study 15
  16. 16. Methodology 16 p p' Object Existence Link Existence Object Subst. LinkEnd Subst. CVLVPs Derivation
 Operation Compilation and Testing CounterexampleVariantSosie [Baudry ISSTA’14] Random operator applied to a random
 code element in the program
  17. 17. Study (the big picture) 17 p' 86 CVL operations Derivation
 Compilation and Testing Object Existence Field Object Existence Interface Object Existence Foreach Object Existence Break … Object Substitution SuperAccess Object Substitution Annotation … Link Existence … LinkEnd Subst … 8 Java projects p'p'p' 370,000 programs %Counterexamples (does not compile) %Variants (only compiles) %Sosies (compiles and pass the test suites)
  18. 18. Subject Programs 18 All have JUnit tests (statement coverage greater than 70%)
  19. 19. Variables and measurements 19 Derivation
 Object Existence Field Object Existence Interface Object Existence Foreach Object Existence Break … Object Substitution SuperAccess Object Substitution Annotation … Link Existence … LinkEnd Subst … %Counterexamples (does not compile) %Variants (only compiles) %Sosies (compiles and pass
  20. 20. Instrumentation 20
  21. 21. Instrumentation (2) 21 370,000 programs one month of computation Compilation and Testing
  22. 22. Results and Findings 22
  23. 23. Results and Findings • 86% of the possible pairs (CVL operator + Type of Java Code Element) resulted in compilable programs at least once ▪Many possibilities to vary a Java program • We found 72 types of CVL-based derivation operations that could produce compilable Java programs – e.g., substituting one child or parent class by another, – suppressing an if statement, – introducing/suppressing a method invocation, – etc. 23
  24. 24. Results and Findings • There are operations that will always lead to counterexamples or to variants 24 Success Rate ~100% ~0%
  25. 25. Results and Findings • Operations with low success rate may not be directly discarded, but specialized ▪Looking into those operations • Qualitatively analysing them with the help of dedicated tooling 25 ~0%
  26. 26. Visualization Tool 26 Project
 View Class
 View Operation
 View 1 2 3
  27. 27. Results and Findings • Specialization to avoid recurrent errors ▪Simple specializations • e.g., removing a try in the case of removing a catch ▪Static-analysis-based specializations • e.g., identifying strongly connected classes to be removed together • Specialized and typed operators ▪Object Existence • Class existence, field existence, parameter existence 27
  28. 28. Results and Findings • Varying entire blocks of code instead of single instructions is more likely to generate correct programs ▪ Do, For, ForEach, While, If, Throw • 70% to 98% of variants when coupled with Object Existence ▪Fine-grained variability works better (ie easier to put variability inside a method than to manipulate coarse elements like « interface ») ▪Anomaly: Classes or Methods à 0.1% – Invoked in other parts of the code 28
  29. 29. Results and Findings • Object Existence is more likely to generate variants ▪ Object Existence à 21.97% of variants ▪ Link Existence à 11.81% of variants ▪ Object Substitution à 9.22% of variants ▪ Link Substitution à 4.51% of variants • Overall CVL is not safe to be directly applied to Java ▪ Many different ways to vary a program • But high probability to break it (90%) ▪ Generic language to vary any base model type but specialization is clearly required 29
  30. 30. Conclusions • Large-scale assessment of derivation operations ▪More than 370,000 derived products • Many kinds of operations ▪86 ways of varying a Java program • Some of them were never considered before by variability studies • Quantitatively and qualitatively supported insights ▪Extensive panorama of success rates for each operation ▪Visualization tool for analyzing the transformations • Open new perspectives for supporting variability in languages, specially in Java 30
  31. 31. Future Work • Using the results to ▪Devise specialized derivation operators for Java ▪Help current variability supporting IDEs to incorporate new possibilities of variation, knowing the risks to do so • Apply derivation operators in different areas, for different objectives (e.g., resilience), such as software diversification 31
  32. 32. 32 Object Existence2 Object Existence3 Object Existence4 Object Existence5 VS1 VS6 VS4 VS5 VS2 0..2 VS3 Object Existence1 >80% <10% >40% Long term vision: variability-aware IDE, (anti-)patterns for any language through automated exploration
 
 Question?

×