1.
Confusion in
Code Reviews
Reasons, Impacts, and
Coping Strategies

2.
Felipe Ebert1
Nicole Novielli2
1 Federal University of Pernambuco, Brazil
2 University of Bari, Italy
3 Eindhoven University of Technology, The Netherlands
Fernando Castor1
Alexander Serebrenik3

3.
Code Review
3

4.
Why Code Reviewing?!
4
Find Defects

5.
Why Code Reviewing?!
5
Find Defects Code Improvement
Alternative Solutions

6.
Why Code Reviewing?!
6
Knowledge Transfer
Find Defects Code Improvement
Alternative Solutions

7.
Why Code Reviewing?!
7
Knowledge Transfer
Team Awareness
Find Defects Code Improvement
Alternative Solutions

8.
Code Reviews Are Not Free!!!
8

9.
9

10.
What is confusion?!
a situation where a person is
uncertain about or unable
to understand something
10

11.
The impacts of confusion!
11

12.
The impacts of confusion!
12

13.
The impacts of confusion!
13

14.
14
Reasons
Impacts
Coping
strategies

15.
Methodology
15
“what developers say” “what developers do”

16.
Survey
• How often do you feel confused...?
• What usually makes you confused...?
• What is the impact of confusion…?
• What do you usually do to overcome
confusion…?
16

17.
Code Review Comments[1]
• Extracted from Android code reviews
• Including expressions of confusion
• 156 general comments
• 151 inline comments
• Discussed and agreed upon by the
authors
17
[1] F. Ebert, F. Castor, N. Novielli, and A.Serebrenik. Confusion Detection in Code
Reviews. In Proceedings of ICSME 2017, pp. 549–553.

18.
Card sorting! Card sorting! Card sorting!
18

19.
What Developers Say?
• 1st survey:
– Android developers
– 17 responses
– Response rate: 0.45%
• 2nd survey:
– Facebook & Twitter
– 24 responses
• 3rd survey:
– Facebook & Twitter
– 13 responses
19

20.
What Developers Say?
• 1st survey:
– Android developers
– 17 responses
– Response rate: 0.45%
• 2nd survey:
– Facebook & Twitter
– 24 responses
• 3rd survey:
– Facebook & Twitter
– 13 responses
20

21.
What Developers Say?
• 1st survey:
– Android developers
– 17 responses
– Response rate: 0.45%
• 2nd survey:
– Facebook & Twitter
– 24 responses
• 3rd survey:
– Facebook & Twitter
– 13 responses
21

22.
We treat survey respondents
as a single population
• 2nd and 3rd
• ANOSIM R = −0.0171 and p-value = 0.542
• ADONIS p-value = 0.975
• 1st and the later two
• ANOSIM R = 0.126 and p-value = 0.01
• ADONIS p-value = 0.191
22

23.
What Developers Do?
• 156 General Comments
• 151 Inline Comments
23

24.
24

25.
30 reasons
14 impacts
13 coping strategies
25

26.
Frequency of Confusion
26
41%

27.
27

28.
28

29.
29

30.
30

31.
31

32.
32

33.
33

34.
34

35.
Concluding Remarks
• Some topics have not been studied yet! 35
• Confusion is present!
– “Developers said!” (survey)
– “Developers did!” (code review comments)
• Confusion in context:
– 30 reasons
– 14 impacts
– 13 coping strategies