Gaeul presented it at ESEC/FSE 2009

1. Bug Triage
with Bug Tossing Graphs
Gaeul Jeong, Sunghun Kim and Thomas Zimmermann
August 26, 2009
Joint meeting of the European Software Engineering Conference (ESEC)
and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE),
Armsterdam,
The Netherlands

2. Bug triage
Bu Bu
g g
X
Fixed
New Assigned Resolved
The life cycle of a bug report (A. Zeller)
2

3. Problem
More than More than
300 reports per day 1,000 developers
Bu
g ? Bu
g
X
Fixed
New Assigned Resolved
3

4. Consequences
Mozilla
297,999 bugs 116,890 bugs
First First
Reported Assignment
Action
26.1 days 161.1 days
(a) All Bugs
pse
131,958 bugs 49,431 bugs
Ecli Reported
144,102 bugs
First
Action 88,706 bugs
First
Assignment
7.1 days First 38.1 days First
Reported Assignment
Action
16.7 days 23.6 days
(b) Verified Bugs
(a) All Bugs
18,498 bugs 15,045 bugs
First First
Reported Assignment
Action
5.2 days 4 19.3 days

5. Bug tossing
Tossing (reassign)
Bu Bu
g g
X
Fixed
New Assigned Resolved
The life cycle of a bug report
5

6. Consequences
%!!
$"!
!"#$%&#'(%)*'+,'-%.#
$!!
#"!
+,-.//0
#!! 12/.345
"!
!
# $ % & " ' ( ) * #!
-,**/0&'1+#2*
6

7. Two Challenges
• Finding appropriate developer is hard
• Wrong assignment(bug tossing) make the
fixing process slow down
7

8. Our solution
• Learning from previous tossing history
• To help
• Assist manual bug triage
• Reducing tossing paths
• Assign bug automatically
8

9. Bug tossing path
• Extracting bug tossing information
• Identify more appropriate developers using
tossing relationship
9

10. Learning from tossing paths
Path tokens
A→D (2), A→E(1)
Extracted path information
B→D(1)
A B C D
A C D E C→D(2), C→E(1)
C E A F D D→E(1)
E→D(1)
F→D(1)
10

11. Learning from tossing paths
Path tokens
Generated graph
A→D (2), A→E(1)
B→D(1)
C→D(2), C→E(1)
D→E(1)
E→D(1)
F→D(1)
11

12. Learning from tossing paths
Path tokens
Generated graph
A→D (2), A→E(1)
A B→D(1)
C→D(2), C→E(1)
D→E(1)
E→D(1)
F→D(1)
11

13. Learning from tossing paths
Path tokens
Generated graph
A→D (2), A→E(1)
A B→D(1)
67%
C→D(2), C→E(1)
D
D→E(1)
E→D(1)
F→D(1)
11

14. Learning from tossing paths
Path tokens
Generated graph
A→D (2), A→E(1)
F A B→D(1)
100% 67% 33%
C→D(2), C→E(1)
D E
100%
100% D→E(1)
67% 33%
C E→D(1)
B
F→D(1)
11

15. Tossing graph from Eclipse
12

16. Manual process with cliques
13

17. Developer structure
14

18. Developer feedback
• “Very neat stuff! The clustering was correct
for the team”
• “This would be useful for both integrators
and managers wanting to understand the
life cycle of bugs”
15

19. Path reduction
• Reducing tossing paths by graph search
• Original path
A B C D
• Recommended path?
A ? Search from A
16

20. Greedy search
for optimal path
• Start from a given node (A)
• Visit heavy neighboring nodes first
F Original : A B C D
A
100% 67% 33%
Our path : A D
D E
100%
17

21. Reduced paths
$
!"#$%&'#$((')**%+,((-#+,'.
#
+,-.//0
" 12/.345
!
" # $ % & ' ( ) !* !! !"
/"%,%+0-((')**%+,(-#+,'.
18

22. Our solution
• Learning from previous tossing history
• To help
• Assist manual bug triage
• Reducing tossing paths
• Assign bug automatically
19

23. Automatic bug triage
Bu
Re
po g
rt
Machine
Machine learner
Who should fix this bug? (J. Anvik)
20

24. Automatic bug triage
Bu
Re
po g
rt
New!
Bu
Re
po g
rt
Machine
Machine learner
Who should fix this bug? (J. Anvik)
20

25. Automatic bug triage
Bu
Re
po g
rt
New!
Bu
Re
po g
rt
Predict
Machine
Machine learner developers
Who should fix this bug? (J. Anvik)
20

26. Experiment
• Integrating bug tossing information
• existing approaches P1 P2 P3
P = [ 1Pp2 , ..., 1 ,np]2 , ..., pn }
{p , = {p p }
• our approach T1 T2 T3
= = , t t 2 t t2 , ..., t }
RPRP [{p1{p1 , p1 , p2 , ..., pn ,pn , ]tn }
21

27. Experiment
• Integrating bug tossing information
• existing approaches ....... First 3 : [p1, p2, p3 ]
P = [ 1Pp2 , ..., 1 ,np]2 , ..., pn }
{p , = {p p }
• our approach .................. First 3 : [p1, t1, p2]
= = , t t 2 t t2 , ..., t }
RPRP [{p1{p1 , p1 , p2 , ..., pn ,pn , ]tn }
22

28. Eclipse triage result
First 2
First 3
Naïve + Tossing
Naïve
First 4
First 5
37.500 45.625 53.750 61.875 70.000 (%)
23

29. Eclipse triage result
First 2
First 3
Naïve + Tossing
Naïve
First 4
First 5
37.500 45.625 53.750 61.875 70.000 (%)
24

30. Mozilla triage result
First 2
First 3
Naïve + Tossing
Naïve
First 4
First 5
25.00 38.75 52.50 66.25 80.00 (%)
25

31. Mozilla triage result
First 2
First 3
Naïve + Tossing
Naïve
First 4
First 5
25.00 38.75 52.50 66.25 80.00 (%)
26

32. Summary
• Tossing graphs help to
• Assist manual bug triage
• Reduce tossing paths
• Assign bug automatically
27

33. Thank you
Gaeul Jeong, Sunghun Kim and Thomas Zimmermann
Joint meeting of the European Software Engineering Conference (ESEC)
and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE),
Armsterdam,
The Netherlands

34. Experiment
• They use the first 165,385 Eclipse bugs for
training
• The machine learners predict developers for
bugs in a testing set, Eclipse bug from 165,397
to 211,822
0 165,397 211,822
Training set Testing set
29

35. ML+
program ML algorithm selection ML only improvement
tossing graph
first2 43.7 44.71 1.01
first3 49.87 53.15 3.27
Naive Bayes first4 56.42 59.95 3.53
first5 60.71 63.48 2.77
Eclipse first2 57.91 58.29 0.38
Bayesian first3 66.71 68.47 1.76
Network first4 69.47 71.48 2.01
first5 75.88 77.14 1.26
first2 33.41 56.39 15.84
first3 45.39 63.82 12.81
Naive Bayes first4 52.94 69.51 12.05
first5 59.35 72.92 11.29
Mozilla first2 40.02 55.85 22.98
Bayesian first3 50.25 63.05 18.43
Network first4 55.4 67.45 16.57
first5 59.53 70.82 13.58
30