This talks introduces you to the Automated Error Reporting feature introduced in Eclipse Mars

1. Because you can’t fix what
you don’t know is broken…

2. About me
2
• Initiator of the Automated Error Reporting Initiative
• Project Lead of Eclipse Code Recommenders
• Eclipse Committer since 2010
• Plug-in Developer for 10+ years (Eclipse 2.1)
• Member of the Eclipse Architecture Council
• Co-Lead of Java User Group Darmstadt
• CEO of Codetrails
• Passion to improve developers day-to-day work with
intelligent and practical tools - mostly using Data
Mining on Big (Software Engineering) Data.
@marcelbruch
marcelbruch

3. Adding a new character to the plot
3
Software
User
{ ——–}
Application
Developer
Product
Manager

4. Imagine the following situation…
4
Software
User
{ ——–}
Application
Developer
Product
Manager

5. But when the user starts the app…
5

6. 6Can you feel their pain…?
This image is not available in your country

7. In the logs you’d find…
7

8. ✘ ✘ ✘
The tragedy of manual error reports
8
✘ ✘
!
maintenance
✘ ✘ ✘
Release SR 2
production
SR 1

9. ✘
!
The tragedy of manual error reports
9
maintenance
Release SR 2
production
SR 1
✘ ✘ ✘

10. 10Not too happy but feeling okay…
This image is not available in your country

11. What can we do?
11
Application Developer
?
{ ——–}

12. The Automated
Error Reporting Initiative
12

13. Live Demo
13

14. About server responses,
Bugzilla, and immediate feedback
„Keep users informed about your progress…“
14

15. Systems Overview
15
AERI Web UI Eclipse BugzillaEclipse IDEs
„Direct Reporter Feedback“

16. Not yet tracked problems…
16

17. Committers needinfo…
17
Status: – – –
Resolution: – – –
Bug 442500 – ….

18. Fixed…
18
Status: – – –
Resolution: – – –
Bug 442500 – ….

19. Won’t Fix…
19
Status: – – –
Resolution: – – –
Bug 442500 – ….

20. Invalid (aka log messages)
20
Status: – – –
Resolution: – – –
Bug 442500 – ….

21. About caching, filtering,
anonymization, and other shortcuts…
„What goes to eclipse.org?“
21

22. Do we send all errors to Eclipse…?
22
really

23. The error reporting funnel
23
✘
structural filter

24. #1: Structural Filtering
24
java.net.UnknownHostException: marketplace.eclipse.org
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
at java.net.Socket.connect(Socket.java:538)
at com.example.HttpClient.openServer(HttpClient.java:527)
at com.example.http.HttpClient.<init>(HttpClient.java:211)
at com.example.HttpURLConnection.connect(HttpURLConnection.java:932)
at org.eclipse.recommenders.internal.news.rcp.PollFeedJob.run(PollFeedJob.java:69)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
Error connecting to ‘http://marketplace.eclipse.org/rss.xml’.

25. #2: Did I already sent this?
25
✘
structural filter
personal filter

26. #3: Did anyone else sent this?
26
✘
structural filter
personal filter
global hist.

27. structural filter
personal filter
If you hit a known issue…
27
✘
global hist.

28. #4: Review
28
review
✘
structural filter
personal filter
global hist.

29. #4: Review
29

30. The server side
Walkthrough through the committer front-end
30

31. The core concepts
31

32. The Core Concepts
32
✘
✘
✘ ✘
✘ ✘
✘
✘✘ Incidents
Problems
Projects

33. The committer front-end
33

34. Creating bug reports
34

35. Creating bug reports
35

36. Notifying committers
36

37. Project Alerts
37

38. Weekly or daily email digests
38

39. Some fun facts
e = mc2 —> errors = (more code)2
39

40. Disclaimer
Don’t draw preliminary conclusions!
Please consult your personal statistican…

41. How many milestones testers?
450 simrel committers 1200 eclipse committers Testers?~7000 error reporters

42. Milestones adoption
42
RC3
RC2
RC1
M7
M6
M5
M4
M3
M2
M1
1000
220
160
650
450
400
560
600
Do new milestones introduce new problems?
70
10
# reporters one week high
SR2
SR1
R
Luna
Mars

43. 0
350
700
1050
1400
4.4.0 4.4.1 4.4.2 M1 M2 M3 M4 M5 M6 M7 RC1 RC2 RC3
Milestones adoption
43
Christmas
Addition to all
EPP packages
RCs
Addition to first
EPP packages

44. How hard do they suffer?
44
300 - 3000 reports
25 testers
50-300 reports
330 testers
3-50 reports
3600 testers
1-2 reports
3000 testers

45. What do they report?
45
0
750
1500
2250
3000
‚Normal‘ Errors
Out of Memory
Freezes
Stack Overflows
3rd Party Involved
Top 100 of 7000
Me

46. What kind of errors are send?
46
3rd Party Involved
16 %
Stack Overflows
0 %
Freezes
9 %
Out Of Memory
1 %
'Normal' Errors
74 %

47. What are the most popular exceptions?
47
32 %
12 %
25 %
3 %
8 %
2 %
18 %

48. What are the most popular exceptions?
48
Other
32 %
No {Class | Method | Field} DefFoundError
12 %
NullPointerException
25 %
IndexOutOfBoundsException
3 %
IllegalArgumentException
8 %
ClassCastException
2 %
{File} IOException
18 %

49. NullPointerExceptions by Projects
49
0
1000
2000
3000
NullPointerException
Code Recommenders
65
Statisticians, note: Projects automatically marked as „being responsible“ fixing these
NullPointerExceptions but does not mean the NPE happened in their code.

50. Tip: Enable null analysis
50
@Override
protected void run(@Nullable List<Incident> slice)
{
for (Incident incident : slice) {

51. JDT offers a lot…
51

52. Tip: Never return null
52
@Override
public Optional<String> getReason() {
return Optional.ofNullable(cause);
}
Use java.util.Optional or com.google.common.base.Optional instead

53. Oh Class, Were Art Thou?
53
Use proper import ranges and define uses on your exports to get rid of
{Class | Method | Field} DefNotFoundErrors, LinkageErrors, and some
ClassCastExceptions. OSGI needs that information to work properly…
Export-Package: org.eclipse.recommenders.utils;
uses:="org.osgi.framework,
org.eclipse.core.runtime,
com.google.common.base,
com.google.common.collect"
Import-Package:
com.google.common.annotations;version="[15.0.0,16.0.0)",

54. Index out of bounds?
54
a b c d e
-1 and array.length do not work. No matter how often you try…
There is no tool support for this. Just unit tests and code that tries to fail gracefully.

55. Tip: If error messages are not helpful…
55
try {
// …
} catch (Exception e) {
log(„An error occurred“, e);
}
21% of all logged errors do not contain stack traces.
try {
// …
} catch (Exception e) {
log(„Invalid argument {}“, value, e);
}

56. Are your users ready for Java 8?
56
both
2,5%
Java 1.7
33%
Java 1,8
64%
Java 1.9
0,3%
Which VM do users run the Eclipse IDE with?

57. Do people anonymize their messages?
57
Sometimes
3 %
No
81 %
Yes
16 %
Are you concerned that your exception message may reveal sensitive data?

58. Bugfixing Rates
Speak plain: Does it work?
58

59. Bugzilla Summary
59
1170 bugs
275 fixed (24%)
100 invalid
70 not eclipse
467 in progress (40%)
172 duplicates (15%)69 works 4 me
28 wont fix
Bugzilla

60. Fixed bugs by projects
60
More (35)
84
---
11
EGit
15
Papyrus
19
Recommenders
41
Xtext
42
Oomph
58

61. In progress & fixed bugs
61
0
30
60
90
120
Oomph TMF Recommenders Papyrus EGit ---
97
1914
17
29
20
11
15
19
4142
58
Fixed
In Progress

62. Future Work?
Feature requests are tracked in bugzilla. Make sure your request is in there and vote for it.
Client:
• Separate view to „review, analyze and send later“
• Making client reusable in other (Eclipse- or SWT-based) applications
• Additional client-side automated analysis
Server:
• Ensure scalability of the system
• Improve duplicate detections
• Go Neon if feedback is good.
Hosting an error reporting service for Java software?
• Need one? TTML.
62

63. How to reach EPP Logging
63
• Code: Check out the EPP Logging Oomph profile
• Wiki: https://wiki.eclipse.org/EPP/Logging
• Manual: https://dev.eclipse.org/recommenders/community/confess/#/about
• Forum: https://www.eclipse.org/forums/index.php/f/69/
• Bugzilla: https://bugs.eclipse.org/bugs/buglist.cgi?component=logging&product=EPP
• Mail: error-reports-inbox@eclipse.org

64. Credits
• Daniel Haftstein, Codetrails – Eclipse client conception and implementation
• Marcel Bruch, Codetrails – Server-side conception and implementation
• Webmasters, Eclipse Foundation – Eclipse server infrastructure & integration
• Thanks to many early adopters and committers providing feedback
64

65. Feedback
65

66. Contact Us
66
Codetrails GmbH
Robert-Bosch-Str. 7
64293 Darmstadt, Germany
www: codetrails.com
phone: 06151 / 2767 092
email: info@codetrails.com
Dr. Marcel Bruch
CEO
mobile: 0179 / 1317 721
email: marcel.bruch@codetrails.com

67. Feedback
67