From Comparison Matrix to
Variability Model
The Wikipedia Case Study
Presented at Automated Software Engineering (ASE’13) conference
Product
Author
First Name
Last Name
Age
Nat.
Ph.D.?
Posit.
Affil.
Spoken
Lang
Nicolas
Sannier
32
French
soon
PhD Student,
Future
Postdoc?
Inria
French,
English,
Reunion
Isl. creole
Mathieu
Acher
29
French
Yes
Associate
Prof.
University
of Rennes
1, Inria,
IRISA
French,
English
Benoit
Baudry
-
French
Yes
Research
Scientist,
Head of
Triskell team
Inria
French,
English

Product Comparison Matrix (PCM)
Product
Author
First Name
Last Name
Age
Nat.
Ph.D.?
Posit.
Affil.
Spoken
Lang
Nicolas
Sannier
32
French
soon
PhD Student,
Future
Postdoc?
Inria
French,
English,
Reunion
Isl. creole
Delayed with a cancelled plane
Mathieu
Acher
29
French
Yes
Associate
Prof.
University
of Rennes
1, Inria,
IRISA
French,
English
Benoit
Baudry
-
French
Yes
Research
Scientist,
Head of
Triskell team
Inria
French,
English
Not available
Compare and Choose your Product Speaker

ASE'2013 - Sannier, Acher and Baudry - From PCM to VM.
-3

ASE'2013 - Sannier, Acher and Baudry - From PCM to VM.
-4

#1 This is a Product Comparison Matrix (PCM)
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM.
-5

#2 This is a Product Comparison Matrix (PCM)
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM.
-6

#3 This is a Product Comparison Matrix (PCM)
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM.
-7

Promises of Product Comparison Matrices (PCMs)
+ Intuitive and easy to
understand
+ Convenient for
comparing, input for
configuring
+ Rich source of
information and
knowledge
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM.
-8

Issues and Challenges
Product Comparison Matrices (PCMs)
- Heterogeneous information
- As the PCM grows up
“more is less”
- Lack of Formalization
- No Automated Support
- Guidance capabilities
- Ad-hoc PCMs
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM.
-9

Issues and Challenges
Product Comparison Matrices (PCMs)
- Heterogeneous information
- As the PCM grows up
“more is less”
- Lack of Formalization
- No Automated Support
- Guidance capabilities
- Ad-hoc PCMs
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM.
- 10

Understanding PCMs
RQ1 What kind information
is presented in PCMs?
Syntax?
Semantics?
Variability patterns?
RQ2 What is the gap between PCMs
and Variability Models?
Related work
• Extensive work on spreadsheets
•
•
But PCMs are specific spreadsheets
Reverse engineering variability models
•
Other artefacts (She et al. ICSE’11, Czarnecki et al. SPLC’07, Abbas et al. CSMR’14)
•
Boolean PCMs (Haslinger et al. FASE’13, Davril et al. FSE’13)
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM.
- 11

The Wikipedia case study
• Open community
• One of the most important analyzable repository of PCMs
• 300+ PCMs
• Multiple domains, multiple concerns, large PCMs
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM.
- 12

Research Methodology
• #0 Extraction of all
381 Wikipedia pages
entitled “Comparison of …”
• #1 A preliminary analysis of some PCMs (variability patterns
definition)
• #2 A Qualitative analysis of randomly selected 50 PCMs
• #3 A Quantitative analysis of all extracted Wikipedia PCMs
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM.
- 13

Qualitative Analysis: 8 Variability Patterns
1. Boolean yes/no answers
2. Partial/constrained yes/no answers
3. Single-value answers
4. Multiple values answers
5. “Unknown” answers
6. Empty cells
7. Inconsistent cells
8. Additional / Extra information
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM.
- 14

Automatically Analysis of 300+ Wikipedia PCMs
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM.
- 15

Variability Patterns: Quantitative Results
Information type
2
3
4
5
6
7
8
Qualitative analysis
47.29%
3.71%
22.75%
4.37%
10.86%
4.83%
0.55%
5.64%
Quantitative analysis
•
1
49.4 %
0.8%
20.4%
15.1%
7.5%
6.8%
-
-
Results
• 75-80% of the PCMs content is
manageable as usual by variability constructs
• 20-25% remaining represent uncertainty or
numerical values
• Calls for more research for modeling and
reasoning about variability
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM.
1. Boolean yes/no answers
2. Partial/constrained yes/no
answers
3. Single-value answers
4. Multiple values answers
5. “Unknown” answers
6. Empty cells
7. Inconsistent cells
8. Additional / Extra information
- 16

Research Directions
Bridging the Gap
between Product Comparison Matrices (PCMs) and Variability Models (VMs)
Contributors (writers)
No more ad-hoc PCMs
PCMs should be easier to create
and maintain
Hopefully a non intrusive solution,
interoperable with Wikipedia
End users (readers)
Manageable information
Better readability
Better services
Developers (readers and writers)
Enabling analysis tools of PCMs
(e.g., synthesis of variability models)
Long term, more global vision:
Generating product comparators and configurators from variability models and PCMs
Note: VMs act as a formal representation of PCMs and intermediates before devising configurators/comparators/*
VMs are not an end-user solution to visualize the PCM
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM.
- 17

Compare and Choose your Answerer!
PCM-driven of course
Look at the Author Comparison Matrix and Choose
Ask your questions ;-)
Product
Author
First Name
Last Name
Age
Nat.
Ph.D.?
Posit.
Affil.
Spoken
Lang
Nicolas
Sannier
32
French
soon
PhD Student,
Future
Postdoc?
Inria
French,
English,
Reunion
Isl. creole
Delayed with a cancelled plane
Mathieu
Acher
29
French
Yes
Associate
Prof.
University
of Rennes
1, Inria,
IRISA
French,
English
Benoit
Baudry
-
French
Yes
Research
Scientist,
Head of
Triskell team
Inria
French,
English
Not available
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM.
- 18