An Empirical Investigation into the Role of API-Level Refactoring during Software Evolution, Miryung Kim, Dongxiang Cai, Sunghun Kim, ICSE' 11: Proceedings of the 2011 ACM and IEEE 33rd International Conference on Software Engineering

1. Miryung Kim1, Dongxiang Cai2, Sunghun Kim2
The University of Texas at Austin1
Hong Kong University of Science & Technology2

2.  It is believed that refactoring improves
software quality and maintainability.
 The role of refactoring has not been
systematically investigated using fine‐
grained revision history at the method level.

3.  We studied the relationships between API‐level
refactorings, bug fixes, the time taken to fix
bugs, and release cycles.
• The number of bug‐fixes increases after API‐level
refactorings.
• A large number of refactoring revisions include
bug fixes or are related to later fixes.
• Bug fix time tends to be shorter after than before.
• API‐level refactorings occur more frequently
before major releases than after.

4.  Motivation & Related Work
 Study Approach Overview
 Research Questions and Results
 Limitations
 Conclusions

5.  Refactoring improves software quality and
maintainability [Opdyke, Griswold, Mens, etc.]
 A lack of refactoring incurs technical debt.
[Cunningham, Lehman]
 Refactor mercilessly [Beck, eXtreme
Programming]
 Productivity improves after re‐architecting a
system [Carriere et al.]
 Defect density decreases after refactoring
[Ratzinger et al.]

6.  The number of bug reports increases after
refactorings [Weißgerber & Diehl]
 Incomplete or inconsistent refactorings cause
bugs [Görg and Weißgeber]
 Code churns are correlated with defect
density [Nagappan & Ball]

7.  Refactoring tools are buggy [Daniel et al.]
 Programmers do not leverage refactoring
tools effectively [Murphy‐Hill et al.]
 API‐level refactorings break client code [Dig
et al.]

8.  Motivation
 Study Approach Overview
 Research Questions and Results
 Limitations
 Related Work
 Conclusions

9.  Q1: Are there more bug fixes after API‐level
refactorings?
 Q2: Are API‐level refactorings correlated to
bug fixes?
 Q3: Do API‐level refactorings reduce the time
taken to fix bugs?
 Q4: Are there relatively fewer API‐level
refactorings before release dates?

10. API‐level
Refactoring Fix Revision
[ICSE 2007,
Reconstruction S. Kim et al.]
[ICSE 2007, M. Kim et al.]
Identify Refactoring Identify
Revisions Fix Revisions
Introducing
Change Distilling
Change [TSE 2007,
[MSR 2005, B. Fluri et al.]
Sliwerski et al.]
Identify Bug Life Time Map line‐level changes
to methods

11. Methods
Change
Add Delete
Revisions
r1 r2 r3 r4 r5 r6 r7 r8

12. API‐level
Refactoring Rename, move, and signature changes
Reconstruction at or above the method header level
[ICSE 2007, M. Kim et al.]
Foo.mA(int)
Foo.mA()
Foo.mB(int)
Foo.mB()
Foo.mC()
Foo.mC()
Bar.mA(long)
Boo.mA(long)
Baz.mA(int)
Boo.mB(long)
Baz.mB(int)

13. Boo.mB(long) => Baz.mB(int)
Foo.mA() => Foo.mA(int)
Foo.mB() => Foo.mB(int)
r1 r2 r3 r4 r5 r6 r7 r8

14. Boo.mB(long) => Baz.mB(int)
Foo.mA() => Foo.mA(int)
Foo.mB() => Foo.mB(int)
r1 r2 r3 r4 r5 r6 r7 r8
Ref Ref

15. Fix Revision Use heuristics to identify bug fix revisions
[ICSE 2007,
e.g., [bug] [id #], [fixed]
S. Kim et al.]
r1 r2 r3 r4 r5 r6 r7 r8
Ref Fix Ref/Fix Fix

16. Introducing
Change Use SVN blame (diff) to track down
[MSR 2005, the changes that led to fixes.
Sliwerski et al.]
Bug‐introducing changes
r1 r2 r3 r4 r5 r6 r7 r8
Ref Fix Ref/Fix Fix

17. Bug life time
Bug‐Introduction Bug‐Fix
r1 r2 r3 r4 r5 r6 r7 r8
Ref Fix Ref/Fix Fix

18. Change Distilling Map line‐level location to a method level location
[TSE 2007, Filter out meaningless changes.
B. Fluri et al.]
r1 r2 r3 r4 r5 r6 r7 r8
Ref Fix Ref/Fix Fix

19. Eclipse JDT jEdit Columba
core
Type IDE Text editor Email client
Period June 5th, 2001 to Sep 2, 2001 to July 9th, 2006 to
May 16th 2007 Apr, 2009 Jan 31st 2010
# revisions 15000 11102 421
# of API‐level 6755 3557 424
refactorings
# of bug fix 3752 1073 150
revisions
# of refactoring 1089 423 36
revisions

20. GT source # Prec. recall
API 14 API‐level refactoring detection 14 0.93 0.93
Refactoring [M. Kim et al. ]
revision
Inspection of change logs only 2 0.50 0.07
Inspection of bug reports only 5 0.75 0.21
Fix 62 Fix‐revision detection [S. Kim 49 0.96 0.76
Revision et al.]
Inspection of change logs only 55 0.92 0.81
Inspection of bug reports only 49 0.94 0.74
Automated techniques vs. Manual Inspection

21.  Motivation and Related Work
 Study Approach Overview
 Research Questions and Results
 Limitations
 Conclusions

22.  Study Method 1: We measure a fix rate (# fix
revisions / K) within K sliding window after
each refactoring revision. K revision windows
Ref Ref Fix Ref Fix Fix Ref Fix

23.  Study Method 1: We measure a fix rate (# fix
revisions / K) within K sliding window after
each refactoring revision. K revision windows
Ref Ref Fix Ref Fix Fix Ref Fix

24.  Study Method 1: We measure a fix rate (# fix
revisions / K) within K sliding window after
each refactoring revision. K revision windows
Ref Ref Fix Ref Fix Fix Ref Fix

25.  Study Method 1: We measure a fix rate (# fix
revisions / K) within K sliding window after
each refactoring revision. K revision windows
Ref Ref Fix Ref Fix Fix Ref Fix

26.  Study Method 1: We measure a fix rate (# fix
revisions / K) within K sliding window after
each refactoring revision. K revision windows
Ref Ref Fix Ref Fix Fix Ref Fix
 Study Method 2: We measure a fix rate at
the method level
M.foo()=>M.bar()
Ref Fix Fix

27.  Results: Fix rates increase after refactorings
at a revision level




average
 
fix rate



       Window size (K)
Refactoring timing

28.  Results: fix rates at the method level also
increase after refactorings
Fix rate at a method level
0.40% 1.00% 8.00%
0.30% 0.80% 6.00%
0.60%
0.20% 4.00%
0.40%
0.10% 0.20% 2.00%
0.00% 0.00% 0.00%
‐30 ‐20 ‐10 0 10 20 30 ‐30 ‐20 ‐10 0 10 20 30 ‐30 ‐20 ‐10 0 10 20 30
Refactoring timing
JDT jEdit Columba
There is a short term increase in the number of bug fixes
after refactorings

29. 20 revision windows
Ref Fix Fix
In Eclipse, 208 refactoring revisions are followed by at least
one bug fix to the same method location within 20 revisions.

30. 20 revision windows
Ref Fix Fix
We randomly sampled 50 revision pairs and manually
inspected them.

31. R F
Types Frequency
1. To fix several related bugs, a developer first refactors 6%
code.
2. Incorrect refactorings cause a bug, inducing a later fix. 6%
3. After a developer performs refactorings, she discovers a 2%
new bug and fixes it.
4 Refactoring with a bug fix is incomplete. A developer 50%
applies a supplementary fix later.
5. A bug fix happened after a refactoring in the same 36%
location, but they are not related.

32.  Study Method 1: We measure the extent of
revisions that include both refactorings and
bug fixes at the same time.
 Study Method 2: We measure the
percentage refactoring revisions that have at
least one bug fix applied to the same method
location within K revisions.

33. Probability of fixes and refactorings at the revision level
Project P(F) P(R) P(R|F) P(R|not F) P(F|R) P(F|not R)
Eclipse 25% 7.3% 12% 5.7% 41.5% 23.7%
JDT
JEdit 9.7% 3.8% 11.5% 3.0% 29.1% 8.9%
Columba 35.7% 8.6% 10.7% 7.4% 44.4% 34.8%

34. Probability of fixes and refactorings at the revision level
Project P(F) P(R) P(R|F) P(R|not F) P(F|R) P(F|not R)
Eclipse 25% 7.3% 12% 5.7% 41.5% 23.7%
JDT
JEdit 9.7% 3.8% 11.5% 3.0% 29.1% 8.9%
Columba 35.7% 8.6% 10.7% 7.4% 44.4% 34.8%
Refactorings and bug fixes often occur together in the same
revisions.

35.  Results:
# of Ref‐revisions that were # of Non‐ref revisions that were
related to fix within (0,K] related to fix within (0,K]
2245
(14.0%)
495 (32.0%)
1053 13847 (86.0%)
(68.0%)
K=20
Refactoring revisions are followed by related bug fixes more
often than non‐refactoring revisions.

36.  Study Method: We estimate the time taken
to fix bugs introduced and resolved near the
refactoring timing.
K revisions K revisions K revisions
refactoring
Bug Introduction Bug Fix revision

37.  Study Method: We estimate the time taken
to fix bugs introduced and resolved near the
refactoring timing.
K revisions K revisions K revisions
AA
refactoring
Bug Introduction Bug Fix revision

38.  Study Method: We estimate the time taken
to fix bugs introduced and resolved near the
refactoring timing.
K revisions K revisions K revisions
AA
BB
refactoring
Bug Introduction Bug Fix revision

39.  Study Method: We estimate the time taken
to fix bugs introduced and resolved near the
refactoring timing.
K revisions K revisions K revisions
AA
BB BA
refactoring
Bug Introduction Bug Fix revision

40.  Study Method: We estimate the time taken
to fix bugs introduced and resolved near the
refactoring timing.
K revisions K revisions K revisions
AA
XB BB BA
refactoring
Bug Introduction Bug Fix revision

41.  Results:
average fix
K=100
time (days)
25 120 250 230.2
19.7 98.6
20 100 200
80 149.6
15 150
60 42.6
10 7.3 100
5.6 40
4.4 21.5
12.2 38.4 48.4
5 20 50
0 0 0
BB AA XB BA BB AA XB BA BB AA XB BA
JDT jEdit Columba
When it comes to fixing bugs introduced near the time of
refactorings, the average fix time tends to decrease after
refactorings

42.  Study Method: We measure refactoring
rates and fix rates before major releases.
K revision windows
Ref Ref Fix Ref Fix Fix Ref Fix
Major Release

43.  Results
30% Eclipse JDT
25%
refactoring rate
20%
15%
10%
5%
0%
K=‐20 K=20 K=‐30 K=30 K=‐40 K=40
window size (K)
There are more refactorings prior to major version releases.

44.  Study intra‐method refactorings and
complex refactorings (e.g., use RefFinder
[Prete et al.])
 Account for the size of edits and the number
of refactorings
 Phase & activity level
 Micro‐commit practices

45.  API‐level refactoring and bug fixes are
correlated temporally and spatially.
 API‐level refactoring serves the role of both
facilitating bug fixes and introducing bugs.
 API‐level refactoring occurs as a part of
behavior correcting edits
Refactoring engines must support safe application of
refactorings and semantic changes together.
The result calls for an in‐depth study on the cost and
benefit of refactorings.

46.  This research is in part supported by National
Science Foundation, CCF‐1043810.
 We thank anonymous reviewers and UIUC
reading group for their comments.