Product Comparison Matrix (PCM), Variability Modeling: The Wikipedia Case Study

1. From Comparison Matrix to Variability Model The Wikipedia Case Study Presented at Automated Software Engineering (ASE’13) conference Product Author First Name Last Name Age Nat. Ph.D.? Posit. Affil. Spoken Lang Nicolas Sannier 32 French soon PhD Student, Future Postdoc? Inria French, English, Reunion Isl. creole Mathieu Acher 29 French Yes Associate Prof. University of Rennes 1, Inria, IRISA French, English Benoit Baudry - French Yes Research Scientist, Head of Triskell team Inria French, English

2. Product Comparison Matrix (PCM) Product Author First Name Last Name Age Nat. Ph.D.? Posit. Affil. Spoken Lang Nicolas Sannier 32 French soon PhD Student, Future Postdoc? Inria French, English, Reunion Isl. creole Delayed with a cancelled plane Mathieu Acher 29 French Yes Associate Prof. University of Rennes 1, Inria, IRISA French, English Benoit Baudry - French Yes Research Scientist, Head of Triskell team Inria French, English Not available Compare and Choose your Product Speaker

3. ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. -3

4. ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. -4

5. #1 This is a Product Comparison Matrix (PCM) ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. -5

8. Promises of Product Comparison Matrices (PCMs) + Intuitive and easy to understand + Convenient for comparing, input for configuring + Rich source of information and knowledge ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. -8

9. Issues and Challenges Product Comparison Matrices (PCMs) - Heterogeneous information - As the PCM grows up “more is less” - Lack of Formalization - No Automated Support - Guidance capabilities - Ad-hoc PCMs ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. -9

10. Issues and Challenges Product Comparison Matrices (PCMs) - Heterogeneous information - As the PCM grows up “more is less” - Lack of Formalization - No Automated Support - Guidance capabilities - Ad-hoc PCMs ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 10

11. Understanding PCMs RQ1 What kind information is presented in PCMs? Syntax? Semantics? Variability patterns? RQ2 What is the gap between PCMs and Variability Models? Related work •  Extensive work on spreadsheets •  •  But PCMs are specific spreadsheets Reverse engineering variability models •  Other artefacts (She et al. ICSE’11, Czarnecki et al. SPLC’07, Abbas et al. CSMR’14) •  Boolean PCMs (Haslinger et al. FASE’13, Davril et al. FSE’13) ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 11

12. The Wikipedia case study •  Open community •  One of the most important analyzable repository of PCMs •  300+ PCMs •  Multiple domains, multiple concerns, large PCMs ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 12

13. Research Methodology •  #0 Extraction of all 381 Wikipedia pages entitled “Comparison of …” •  #1 A preliminary analysis of some PCMs (variability patterns definition) •  #2 A Qualitative analysis of randomly selected 50 PCMs •  #3 A Quantitative analysis of all extracted Wikipedia PCMs ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 13

14. Qualitative Analysis: 8 Variability Patterns 1.  Boolean yes/no answers 2.  Partial/constrained yes/no answers 3.  Single-value answers 4.  Multiple values answers 5.  “Unknown” answers 6.  Empty cells 7.  Inconsistent cells 8.  Additional / Extra information ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 14

15. Automatically Analysis of 300+ Wikipedia PCMs ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 15

16. Variability Patterns: Quantitative Results Information type 2 3 4 5 6 7 8 Qualitative analysis 47.29% 3.71% 22.75% 4.37% 10.86% 4.83% 0.55% 5.64% Quantitative analysis •  1 49.4 % 0.8% 20.4% 15.1% 7.5% 6.8% - - Results •  75-80% of the PCMs content is manageable as usual by variability constructs •  20-25% remaining represent uncertainty or numerical values •  Calls for more research for modeling and reasoning about variability ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. 1.  Boolean yes/no answers 2.  Partial/constrained yes/no answers 3.  Single-value answers 4.  Multiple values answers 5.  “Unknown” answers 6.  Empty cells 7.  Inconsistent cells 8.  Additional / Extra information - 16

17. Research Directions Bridging the Gap between Product Comparison Matrices (PCMs) and Variability Models (VMs) Contributors (writers) No more ad-hoc PCMs PCMs should be easier to create and maintain Hopefully a non intrusive solution, interoperable with Wikipedia End users (readers) Manageable information Better readability Better services Developers (readers and writers) Enabling analysis tools of PCMs (e.g., synthesis of variability models) Long term, more global vision: Generating product comparators and configurators from variability models and PCMs Note: VMs act as a formal representation of PCMs and intermediates before devising configurators/comparators/* VMs are not an end-user solution to visualize the PCM ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 17

18. Compare and Choose your Answerer! PCM-driven of course Look at the Author Comparison Matrix and Choose Ask your questions ;-) Product Author First Name Last Name Age Nat. Ph.D.? Posit. Affil. Spoken Lang Nicolas Sannier 32 French soon PhD Student, Future Postdoc? Inria French, English, Reunion Isl. creole Delayed with a cancelled plane Mathieu Acher 29 French Yes Associate Prof. University of Rennes 1, Inria, IRISA French, English Benoit Baudry - French Yes Research Scientist, Head of Triskell team Inria French, English Not available ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 18