How Do Developers Toggle Breakpoints: Observational Studies

1. How Do Developers Toggle Breakpoints?
– Observational Studies –
Fabio Petrillo, Hyan Mandian, Aiko Yamashita, Foutse Khomh, Yann-Gaël Guéhéneuc
Polytechnique Montréal, Canada
UniRitter, Brazil
Oslo and Akershus University College of Applied Sciences, Norway
IEEE International Conference on Software Quality, Reliability and Security (QRS’2017)

2. Is debugging important?
Developers spend over two-thirds of their time (68%) investigating code,
and the majority of this time is spent debugging code (33%) [LaToza
2010].
LaToza, T. D., & Myers, B. a. (2010). Developers ask reachability questions. 2010 ACM/IEEE 32nd
International Conference on Software Engineering, 1, 185–194.
2

3. Is debugging important?
● Developers spend over two-thirds of their time (68%) investigating
code, and the majority of this time is spent debugging code (33%)
● 40% of the developers go to debug mode directly after they have
performed a change
● 63% of the developers used dynamic approaches to perform impact
analysis over their changes.
LaToza, T. D., & Myers, B. a. (2010). Developers ask reachability questions. 2010 ACM/IEEE 32nd
International Conference on Software Engineering, 1, 185–194.
S. Jiang, C. McMillan, and R. Santelices, “Do programmers do change impact analysis in debugging?”
Empirical Software Engineering, pp. 1–39, 2016. 3

4. During interactive debugging, lonely developers
navigate the source code
step into many statements
traverse dozens of method invocations
gain understanding about the system.
However, first, they have to SET a breakpoint.
4

5. Setting a breakpoint
● Setting a breakpoint is one of the most
frequently used features of IDEs
● To decide where to set a breakpoint, developers:
○ must use their observations
○ recall their experiences with similar tasks
○ formulate hypotheses about the task
5

6. Setting a breakpoint
● Tiarks and Röhms observed that developers have
difficulties in finding locations for setting the
breakpoints
● Suggesting that this is a difficult activity
Supporting developers to set appropriate
reakpoints could reduce debugging effort
R. Tiarks and T. Rö hm, “Challenges in Program Comprehension,” Softwaretechnik-Trends, vol. 32, no. 2,
pp. 19–20, May 2013. 6

7. However, no study has been performed on
how developers set breakpoints.
7

8. Study Design
8

9. Study design
● We analysed 45 video-recorded debugging
sessions
● Extracting breakpoint data
● 2 independent studies
○ Study 1: JabRef (own videos)
○ Study 2: PdfSam and Raptor (videos from
study Jiang et al. [3]
9

10. Study 1 - JabRef
● 20 developpers (8 freelancers and 12 students)
● 5 true tasks from JabRef issue system (GitHub)
● Performing debugging to locate the faults
● Maximum limit of one-hour per task
● Debugging data (breakpoints, stepping, method invocations)
were automatically collected by a Eclipse tracing plug-in.
● On-line post-experiment questionnaire to collect information
about the study
10

11. Study 2 - PdfSam and Raptor
● Re-analysis of 20 videos of debugging sessions available from
a study conducted by Jiang et al. [3]
● Study on change impact analysis
● One defect per system
● Fault correction tasks
11

12. Summary of Studies
● 3 different systems
● 7 different tasks
○ 5 bug locations
○ 2 bug corrections
● 307 breakpoints
● More than 10 hours of video
12

13. Results
13

14. RQ1: What is the effort (time) for setting the first breakpoint in relation to
the total effort for a debugging task?
Study 1 : in average participants spend 27% of the total
task duration to set the first breakpoint (std. Dev. 17%)
Study 2 : in average participants spend 23% of the total
task duration to set the first breakpoint (std. Dev. 17%)
14

15. We conclude that the effort for setting the first breakpoint takes
near one quarter of the total effort of
a single debugging session. This effort is important
and hints that debugging time could be reduced by
providing tool support for setting breakpoints.
15

16. RQ2: Is there a correlation between time of first breakpoint
and task’s elapsed time?
● there is a clear correlation (ρ = −0.47)
16

17. 17

18. If developers toggle breakpoints
carefully, they complete tasks faster
than developers who toggle breakpoints
too quickly
18

19. RQ3: Are there consistent, common debugging trends to the types of
statements among developers?
19
Study 1 Study 2
Breakpoints per type of statement

20. RQ3: Are there consistent, common debugging trends to the types of
statements among developers?
20
Study 1 Study 2
Breakpoints per type of statement

21. Yes, there are trends on developers
chose a breakpoint. 50% of the
breakpoints were set on call statements
while loop statement the least common
(2-4%)
21

22. RQ4: Are there consistent, common debugging trends for
setting breakpoints on the same line, method, or class,
among developers?
22

23. 23

24. 24
Study 1 - JabRef
Breakpoints in the same line of code
Study 2 - PdfSam
Study 2 - Raptor

25. 25
Study 1 - Breakpoints by class across different tasks

26. 26
Study 1 - Methods with 5 or more breakpoints

27. Developers do not choose breakpoints lightly, but
there is a rationale in their setting breakpoints,
because different developers set breakpoints on the
same line of code for the same task and
different developers set breakpoints on the same type
or method for different tasks.
27

28. This results shows the usefulness of collecting and
sharing breakpoints to assist developers during
maintenance tasks, showing an opportunity
to recommend those locations as candidates for new
debugging sessions.
28

29. ● Setting the first breakpoints is hard (~ 25% of task time)
● ~ 50% of the breakpoints were set on call statements
● When developers toggle breakpoints carefully, they complete tasks faster than
developers who toggle breakpoints too quickly
● Different developers set breakpoints on the same line of code for the same task
● Developers need tools that can assist them in locating adequate places to set
breakpoints in the code and our observations suggest the opportunity for a
breakpoint recommendation system
● Our work helps to build a grounded theory on the setting of breakpoints to
improve debuggers and other tool
● fill the gap in the literature about interactive debugging behaviors
● There is a lot of investigating about interactive debugging phenomena!
Final Remarks
29

30. IEEE International Conference on Software Quality, Reliability and Security (QRS’2017)
Thank you!
@petrillofabio
fabio@petrillo.com