This document discusses a study on predicting the compatibility of Eclipse third-party plugins (ETPs) with new Eclipse releases. The researchers built prediction models using logistic regression based on ETPs' usage of Eclipse APIs versus non-APIs. They trained and tested 36 models and found high accuracy, precision, and recall over 80% in many cases. The models show promise for predicting compatibility and could be developed into a tool to help ETP users and developers.
5. Motivation
• Previous study (Survival of ETPs – ICSM 2012), we
tested compatibility of 345 ETP-APIs and 288 ETP-non-
APIs with different Eclipse releases.
• Our observations:
1. ETP-APIs always compatible in new Eclipse releases.
2. Bad interfaces are the main cause of incompatibilities.
3. Informally, found old bad interfaces stable.
• Formally verified observation 2.
• Trained prediction models
• Tested the prediction models.
Software Engineering and Technology (SET) PAGE 49-6-2014
6. Motivation
• Previous study (Survival of ETPs – ICSM 2012), we
tested compatibility of 345 ETP-APIs and 288 ETP-non-
APIs with different Eclipse releases.
• Our observations:
1. ETP-APIs always compatible in new Eclipse releases.
2. Bad interfaces are the main cause of incompatibilities.
3. Informally, found old bad interfaces stable.
• Formally verified observation 2.
• Trained prediction models
• Tested the prediction models.
Software Engineering and Technology (SET) PAGE 59-6-2014
7. Compatibility prediction
• Requirements of compatibility
prediction:
− Current SDK compatible with ETP
− Later SDK to make prediction
• We built 36 prediction models in total
• Models are bases on bad interfaces
used by ETPs
Software Engineering and Technology (SET) PAGE 79-6-2014
10. Model Training Example
In put
P1 P2 P3
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
24-09-2012 10
11. Model Training Example
In put
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
P1 P2 P3
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
24-09-2012 11
12. Model Training Example
In put
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
P1 P2 P3
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
1 0 0
24-09-2012 12
13. Model Training Example
In put
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
P1
15
P2
4
P3
8
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
1 0 0
24-09-2012 13
14. Model Training Example
In put
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
P1
15
P2
4
P3
8
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
1 0 0
24-09-2012 14
15. Model Training Example
In put
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
P1
15
P2
4
P3
8
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
1 0 0
24-09-2012 15
Dependent variable
Independent variables
16. Model Training Example
In put
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
P1
15
P2
4
P3
8
Logistic
Regression
Machine
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
1 0 0
24-09-2012 16
Dependent variable
Independent variables
17. Model Training Example
In put
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
P1
15
P2
4
P3
8
Logistic
Regression
Out put
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
𝑏2𝑏1 𝑏3 𝑏4
Machine
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
1 0 0
24-09-2012 17
Dependent variable
Independent variables
18. Model Training Example
In put
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
P1
15
P2
4
P3
8
Logistic
Regression
Out put
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
𝑏2𝑏1 𝑏3 𝑏4
Machine
𝑷 𝑋 = 1/(1 + 𝑒−𝒛
)
𝒛 = 𝑏0 + 𝑏1 ∗ 𝑺𝑫𝑲1.0 + 𝑏4 ∗ 𝑺𝑫𝑲3.0
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
1 0 0
24-09-2012 18
Dependent variable
Independent variables
19. Model Training Example
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
P4
20
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
24-09-2012 19
20. Model Training Example
1.
0
1.0 2.0 2.1 3.0
Eclipse 3.0 non-APIs
P4
20
ETPs supported in Eclipse 3.0 – Prediction in Eclipse X, where X > 3.0
24-09-2012 20
𝑷 𝑋 = 1/(1 + 𝑒−𝒛)
𝒛 = 𝑏0 + 𝑏1 ∗ 8 + 𝑏4 ∗ 5
𝑷 𝑋 <0.5 – incompatibility
𝑷 𝑋 >=0.5 – compatibility
21. Results
• In both model training and testing: High
Precision, Accuracy, and Recall, where
some were 80% and more
24-09-2012 21
Model Testing Error Analysis
3.5 3.6 3.7
A P R A P R A P R
3.4 94 100 94 93 100 93 93 100 93
3.5 91 94 96 88 91 96
22. Conclusion and Future Work
• Mining interface usage from ETPs
to detect or predict compatibility
shows good results.
• Next, develop a domain specific
tool to make predictions.
• Who can use the tool? users and
developers of ETPs
•
Software Engineering and Technology (SET) PAGE 229-6-2014
23. Thank you for listening
Software Engineering and Technology (SET) PAGE 239-6-2014
Notes de l'éditeur
Technical support from framework developers, ease of maintenance, and