How to fight bugs in legacy code? Should you do it at all? What to do if there are hundreds or even thousands of errors?(that’s usually the case) How to avoid spending a plethora of man-hours on this? And still, how did you work with Unreal Engine?

1. How to Fix Hundreds of Bugs in
Legacy Code and Not Die
(Unreal Engine 4)
George Gribkov

2. About Me:
George Gribkov
A C++ developer and developer of the PVS-
Studio static code analyzer
I help our clients fight errors
I write articles on looking
for bugs in open-source
projects.
gribkov@viva64.com
2

3.  How to fight bugs in legacy code?
Overview
3

4.  How to fight bugs in legacy code?
 Should you do it at all?
Overview
4

5.  How to fight bugs in legacy code?
 Should you do it at all?
 What to do if there are hundreds or even thousands
of errors?
Overview
5

6.  How to fight bugs in legacy code?
 Should you do it at all?
 What to do if there are hundreds or even thousands
of errors?
(that’s usually the case)
Overview
6

7.  How to fight bugs in legacy code?
 Should you do it at all?
 What to do if there are hundreds or even thousands of errors?
(that’s usually the case)
 How to avoid spending a plethora of man-hours on this?
Overview
7

8.  How to fight bugs in legacy code?
 Should you do it at all?
 What to do if there are hundreds or even thousands of errors?
(that’s usually the case)
 How to avoid spending a plethora of man-hours on this?
 And still, how did you work with Unreal Engine?
Overview
8

9. Fighting Bug in Legacy Code
9

10. Should You Fight Bugs in Legacy Code?
10

11. 11
YES!

12.  Old bugs are still bugs
 Old vulnerabilities are still vulnerabilities
 A charged gun will shoot you in the foot
eventually
Why?
12

13. 13
Ariane 4

14. 14
14
Ariane 4 113 successful launches

15. 15
15
Ariane 5

16. 16
16
Ariane 5
370 000 000 $ blown up!

17.  Unit testing
 Integration testing
 System testing
 …
Ways to Look for Bugs
17

18.  Unit testing
 Integration testing
 System testing
 …
 Dynamic analysis
 Static analysis
Ways to Look for Errors
18

19.  Unit testing
 Integration testing
 System testing
 …
 Dynamic analysis
 Static analysis
Ways to Look for Errors
19

20. Static analysis is an automatic code review
procedure.
What is Static Analysis?
20

21. How Does It Help?
21

22. How Does It Help?
22
Static analysis

23. 23
Wait, Wait…

24. Oh No!
24
Legacy’s somewhere here!!!

25. 1. Set up continuous analysis
2.
3.
What’s a Good Way to Deal With Old Bugs?
25

26. 1. Set up continuous analysis
2. Protect yourself from new bugs
3.
What’s a Good Way to Deal With Old Bugs?
26

27. 1. Set up continuous analysis
2. Protect yourself from new bugs
3. Gradually go through the analyzer’s report line-
by-line, making sure to spend few resources
each time.
What’s a Good Way to Deal With Old Bugs?
27

28. How Our Two Developers Fixed 2000 Bugs, That Our
Analyzer Reported in Unreal Engine 4, in 17 Business
Days
28

29. 29
How It Started

30.  My boss found errors in UE 4
and wrote an article
 Developers from Epic Games
liked it a lot
 They wanted to fix more bugs
and trusted that matter to us
How It Started
30

31. 31
Take One
(Or How We Got Our Asses Kicked)

32.  The most convenient approach: check the
project through Visual Studio
 Luckily, UE has a set of scripts to generate
.vcxproj files
The First Check
32

33. 1.Generate project files
2.Build the project
3.Start the analysis from Visual
Studio
4.???????
5.
The First Analysis Attempt
33

34. 1.Generate project files
2.Build the project
3.Start the analysis from Visual
Studio
4.???????
5.EPIC GAMES FAIL
The First Analysis Attempt
34

35.  Generated project files are just a wrapper
 These wrappers call the Unreal Build Tool
 Unreal Build Tool starts cl.exe (or clang, for Linux
builds)
How the Unreal Engine Project Is Built
35

36.  Generated project files are just a wrapper
 These wrappers call the Unreal Build Tool
 Unreal Build Tool starts cl.exe (or clang, for Linux
builds)
 Because of all these layers the analyzer cannot
collect the parameters required for compilation
How Unreal Engine Project Is Built
36

37. 37
Take Two
(Or How We Did Everything Right)

38.  What if we try to catch compiler calls directly?
The Second Check
38

39.  What if we try to catch compiler calls directly?
 It’s pretty fortunate that we have a special utility
that can monitor compilation
The Second Check
39

40. The Second Analysis Attempt
40
1. Start the compilation monitoring
utility before building the project
2. The utility collects all the required
data
3. Right after that’s done, the analysis
starts
4. ???????
5.

41. 1. Start the compilation monitoring
utility before building the project
2. The utility collects all the required
data
3. Right after that’s done,
the analysis starts
4. ???????
5. EPIC WIN!!!
The Second Analysis Attempt
41

42. 42
Analysis Results

43. Analysis Results
43
 1192 top-level warnings
(Level 1)
 629 second-level warnings
(Level 2)
 1821 warnings in total
(Level 3 not included)

44. 44
44

45. Why Is It Typical to Get Thousands of Warnings?
45
 Notepad++ – 5 241 warnings

46. Why Is It Typical to Get Thousands of Warnings?
46
 WinMerge – 8 786 warnings

47. Why Is It Typical to Get Thousands of Warnings?
47
 Media Player Classic Home Cinema – 33 370 warnings

48. 48
So What’s Next?

49. How to Process Old Errors
49
1. Set up your development process so that it involves
continuous analysis

50. 1. Continuous Analysis
50

51. 1. Continuous Analysis
51

52. 1. Continuous Analysis
52

53. 1. Continuous Analysis
53

54. 1. Continuous Analysis
54

55. 1. Continuous Analysis
55

56. 1. Continuous Analysis
56

57. 1. Continuous Analysis
57

58. How to Process Old Errors
58
1. Set up your development process so that it involves
continuous analysis
2. Prevent new errors

59. 2. Prevent New Errors
59
Статанализ

60. 2. Prevent New Errors
60
 How to detect new triggerings among old
warnings?
 Approach 1: incremental analysis
 Approach 2: warning mass suppression (the use
of a suppress base)

61. How to Work with a Suppress Base
61
 Hide all old warnings
 Get 0 warnings after analysis
 Starting from this moment, all new mistakes
will be detected at once

62. How to Work with a Suppress Base
62

63. How to Work with a Suppress Base
63

64. How to Work with a Suppress Base
64
Old warnings don’t go anywhere!
We just separated them from the new ones and
work on them separately.

65. Incremental Analysis or a Suppress Base?
65
 When dealing with Unreal Engine 4, we used
only incremental analysis
 Suppress bases are more convenient if you are
the one developing the project
 You can use both approaches simultaneously

66. How to Process Old Errors
66
1. Set up your development process so that it involves
continuous analysis
2. Prevent new errors
3. Process the analyzer’s report gradually, line-by-line,
and save resources.

67. 3. How to Work with a Huge Report
67
Two approaches:
 The Pareto principle (80/20)
 The ratchet method

68. The Pareto principles
68

69. The Pareto Principle
69
 In order to fix 80% of errors, reviewing 20% of
warnings is sufficient
 The warnings have already been separated into
these portions and are ready for use

70. The Pareto Principle
70
 Media Player Classic Home Cinema – 33 370 warnings

71. The Pareto Principle
71
 33 370 warnings –> 1 935 warnings

72. 20% of warnings in Unreal Engine
72
 1192 top-level warnings
(Level 1)
 629 second-level warnings
(Level 2)
 1821 warnings total
(Level 3 not included)

73. The Ratchet Method
73

74.  The total number of warnings is committed to
the version control system
 Changes are allowed only if they do not add to
this number of errors
The Ratchet Method
74

75. How the Ratchet Method Works
75

76. How the Ratchet Method Works
76

77.  Ivan Ponomaryov — Continuous Static Code
Analysis
A Talk on Related Topic
77

78. 78
Warning Number
Graphs

79.  Expectation
Warning Number Graph
0
5
10
15
20
25
1 2 3 4 5
Warnings
79

80.  Expectation
Warning Number Graph
80
0
5
10
15
20
25
1 2 3 4 5
Warnings
0
5
10
15
20
25
1 2 3 4 5
Warnings
 Reality

81. Progress for Unreal Engine 4
81

82. A Small Dessert
82
 After fixing all errors we found two more:

83. A Sample Graph for the Ratchet Method
83

84.  Developers from Epic Games were happy
 They now regularly use static analysis
 As for us… we wrote another article :)
Recap
84

85. Conclusion
85

86. 86
Want to Clean Up Legacy Code?
Use Static Analysis!

87. If the Warnings Count in Thousands:
87
1.Set up your development process so that it involves
continuous analysis
2.Prevent new mistakes
3.Work through the analyzer’s report gradually
(the Pareto and ratchet principles)

88. Try Static Analysis!

89. Try Static Analysis!
e-mail:
gribkov@viva64.com

90. Try Static Analysis!
e-mail:
gribkov@viva64.com
Q&A