A keynote delivered for the 2020 Workshop on Machine Learning Techniques for Software Quality Evolution March 20, 2018 | co-located with SANER 2018, Campobasso, Italy https://maltesque.github.io/index.html Abstract - Industry 4.0 is the current trend of automation and data exchange in manufacturing technologies. This trend forces the manufacturing industry to switch to a more agile way of working, hence software engineering teams can and should take a leading role therein. This talk will explore the state-of-the-art in agile software development and the opportunities this may present for Industry 4.0. Consequently it will address questions like: Will our test suite detect critical defects early ? Where should we fix a defect? How long will it take to fix defects? Which team members get frustrated ? Can we use bots to process easy issues?

1. Agile Software Engineering
—
Opportunities for Industry 4.0
Prof. Serge Demeyer
Keynote at MaLTeSQuE, March 2018
(Workshop on Machine Learning Techniques for Software Quality Evolution)

2. Books
• 2 books
• 3 proceedings (editor) Best Teacher’s Award
Top Publications
Spin Off &
Start Up

3. Agile Software Engineering
—
Opportunities for Industry 4.0

4. Smart
Logistics
Smart
Agriculture

5. Industry 4.0 Internet of Things

6. variability
"lot size 1"
safety
IEC 62061 (mechatronics)
RTCA/DO-178C (avionics)
IEC 62304 (medical)
ISO 26262 (automotive)
agility
rapid customer feedback
faster release cycles

7. Six decades into the computer revolution, four
decades since the invention of the microprocessor,
and two decades into the rise of the modern Internet,
all of the technology required to transform industries
through software finally works and can be widely
delivered at global scale.

8. Agile Software Engineering
—
Opportunities for Industry 4.0

9. Continuous Integration / Deployment
<<Breaking the Build>>
version
control
build
developer
tests
deploy
scenario
tests
deploy to
production
measure &
validate

10. [Khom2014] Khomh, F. Adams, B, Dhaliwal, T and Zou, Y Understanding
the Impact of Rapid Releases on Software Quality: The Case of Firefox,
Empirical Software Engineering, Springer. http://link.springer.com/article/
10.1007/s10664-014-9308-x
1.0 1.5 2.0 3.0 3.5 3.6 4.0 5.0 7.0
8.0
9.0
Traditional Release Cycle Rapid Release Cycle
(a) Time Line of Major Versions of FireFox
(b) Time Line of Minor Versions of FireFox
Figure 1. Timeline of FireFox versions.
channels are respectively 100,000 for NIGHTLY, 1 million
for AURORA, 10 million for BETA and 100+ millions for
a major Firefox version [11]. NIGHTLY reaches Firefox
developers and contributors, while other channels (i.e., AU-
RORA and BETA) recruit external users for testing. The
source code on AURORA is tested by web developers who
are interested in the latest standards, and by Firefox add-on
developers who are willing to experiment with new browser
APIs. The BETA channel is tested by Firefox’s regular beta
by bug triaging developers and assigned for fixing. When
a developer fixes a bug, he typically submits a patch to
Bugzilla. Once approved, the patch code is integrated into
the source code of Firefox on the corresponding channel and
migrated through the other channels for release. Bugs that
take too long to get fixed and hence miss a scheduled release
are picked up by the next release’s channel.
III. STUDY DESIGN

11. [Khom2014] Khomh, F. Adams, B, Dhaliwal, T and Zou, Y Understanding
the Impact of Rapid Releases on Software Quality: The Case of Firefox,
Empirical Software Engineering, Springer. http://link.springer.com/article/
10.1007/s10664-014-9308-x
✓ bugs are fixed faster
(but … harder bugs propagated to later releases)
✓ amount of pre- & post-release bugs ± the same
✓ the program crashes earlier
(perhaps due to recent features)
3.6 4.0 5.0 7.0
8.0
9.0
Rapid Release Cycle
rs and assigned for fixing. When
he typically submits a patch to
the patch code is integrated into
on the corresponding channel and
er channels for release. Bugs that
nd hence miss a scheduled release
release’s channel.
TUDY DESIGN
earch questions:
ease cycle affect the
erence in the number
control for the time
lease dates. However,
tly lower for versions
les, i.e., failures seem
cycle affect the fixing
ter for versions devel-
e cycle affect software
d release model are
.e., the proportion of
ersions that possibly
5.0 NIGHTLY 6.0 NIGHTLY 7.0 NIGHTLY 8.0 NIGHTLY
5.0 AURORA 6.0 AURORA 7.0 AURORA
5.0 BETA 6.0 BETA
5.0 MAIN
New Feature Development
6 Weeks 6 Weeks 6 Weeks 6 Weeks
Figure 2. Development and Release Process of Mozilla Firefox
major release was made. Figure 1(b) shows the release dates
of the minor versions of Firefox.
With the advent of shorter release cycles in March 2011,
new features need to be tested and delivered to users faster.
To achieve this goal, Firefox changed its development pro-
cess. First, versions are no longer supported in parallel, i.e.,

12. Version Control
• CVS, Subversion, Git, …
• Rational ClearCase
• Perforce,
• Visual Source Safe
• …
Issue Tracking
• Bugzilla
• BugTracker.NET
• ClearQuest
• JIRA
• Mant
• Visual Studio Team Foundation
Server
• …
Automate the Build
• make
• Ant, Maven
• MSBuild
• OpenMake
• Build Forge
• …
Automated Testing
• HP QuickTest Professional
• IBM Rational Functional Tester
• Maveryx
• Selenium
• TestComplete
• Visual Studio Test Professional
Microsoft 2010
• …
… mailing archives, newsgroups, chat-boxes, facebook, twitter, …
All of a sudden empirical research has
what any empirical science needs: a
large corpus of objects to analyze.
[Bertrand Meyer's technology blog]
Software Repositories & Archives

13. 12. Conclusion
Mining Software Repositories
13
Conferences
• 2018—15th edition, Gothenburg, Sweden
• 2017—14th edition, Buenos Aires, Argentina
• 2016—13th edition, Austin, Texas
• 2015—12th edition, Florence, Italy
• 2014—11th edition, Hyderabad, India
• 2013—10th edition, San Francisco, USA
• 2012—9th edition, Zürich, CH
• 2011—8th edition, Honolulu, HI, USA
• 2010—7th edition, Cape Town, ZAF
• 2009—6th edition, Vancouver, CAN
• 2008—5th edition, Leipzig, DEU
• 2007—4th edition, Minneapolis, MN, USA
• 2006—3rd edition, Shanghai, CHN
• 2005—2nd edition, Saint Luis, MO, USA
• 2004—1st edition, Edinburgh, UK
The Mining Software Repositories (MSR) field analyzes the rich
data available in software repositories to uncover interesting and
actionable information about software systems and projects.
Hall of Fame—Mining Challenge
• 2018 — IDE Event Stream (JetBrains)
• 2017 — TravisTorrent (Github)
• 2016 — BOA (SourceForge & Github)
• 2015 — StackOverflow
• 2014 — GitHub
• 2013 — StackOverflow
• 2012 — Android
• 2011 — Netbeans+Eclipse
• 2010 — GNOME Projects
• 2009 — GNOME project
• 2008 — Eclipse
• 2007 — Eclipse Developer
• 2006 — PostgreSQL

14. Agile Software Engineering
—
Opportunities for Industry 4.0

15. Mutation Testing
mutators
&&
++
>
…
||
- -
>=
…
public static int search(int key, int[] a) {
int lo = 0;
int hi = a.length - 1;
while (lo <= hi) {
// Key is in a[lo..hi] or not present.
int mid = lo + (hi - lo) / 2;
if (key < a[mid]) hi = mid - 1;
else if (key > a[mid]) lo = mid + 1;
else return mid;
}
return -1;
}
source code
+
public static int search(int key, int[] a) {
int lo = 0;
int hi = a.length - 1;
while (lo <= hi) {
// Key is in a[lo..hi] or not present.
int mid = lo + (hi - lo) / 2;
if (key <= a[mid]) hi = mid - 1;
else if (key > a[mid]) lo = mid + 1;
else return mid;
}
return -1;
}
public static int search(int key, int[] a) {
int lo = 0;
int hi = a.length - 1;
while (lo <= hi) {
// Key is in a[lo..hi] or not present.
int mid = lo + (hi - lo) / 2;
if (key < a[mid]) hi = mid - 1;
else if (key >= a[mid]) lo = mid + 1;
else return mid;
}
return -1;
}
mutant1 mutant 2

16. Branch Coverage vs.
Mutation Coverage
. ANALYSIS OF THE RESULTS 33
gure 5.1: Mutation coverage results of classes in Segmentation component
ted in decreasing order
Figure 5.1: Mutation coverage results of classes in Segmentation
sorted in decreasing order
Unit tests only !
— TESTOMAT Project

17. Data mining
inside !
Spectrum Based Fault Localisation
wastedeffort
7
46
56
122
48
2
104
124
74
6
109
9
83
68
116
90
86
27
32
97
128
16
115
81
52
57
60
42
62
63
69
75
23
26
88
65
77
84
73
1
133
51
113
76
117
53
61
38
34
105
29
72
78
96
43
103
33
4
87
49
30
85
15
112
25
118
125
82
20
110
132
39
45
89
71
10
114
5
24
64
35
54
40
3
127
70
28
120
98
121
37
19
100
11
99
108
14
17
131
50
95
66
58
41
80
92
93
47
44
91
67
8
126
31
36
79
55
13
21
22
106
119
123
130
102
111
129
12
18
101
94
107
59
7
46
56
122
48
2
104
124
74
6
109
9
83
68
116
90
86
27
32
97
128
16
115
81
52
57
60
42
62
63
69
75
23
26
88
65
77
84
73
1
133
51
113
76
117
53
61
38
34
105
29
72
78
96
43
103
33
4
87
49
30
85
15
112
25
118
125
82
20
110
132
39
45
89
71
10
114
5
24
64
35
54
40
3
127
70
28
120
98
121
37
19
100
11
99
108
14
17
131
50
95
66
58
41
80
92
93
47
44
91
67
8
126
31
36
79
55
13
21
22
106
119
123
130
102
111
129
12
18
101
94
107
59
11513515517519511151140116511901
11513515517519511151140116511901
Patterned Spectrum Analysis
Raw Spectrum Analysis
wastedeffort
3
15
2
8
9
23
1
14
17
5
18
16
4
7
26
21
25
22
6
13
24
10
12
19
20
27
3
15
2
8
9
23
1
14
17
5
18
16
4
7
26
21
25
22
6
13
24
10
12
19
20
27
151101151201251301351401451
151101151201251301351401451
Patterned Spectrum Analysis
Raw Spectrum Analysis
In ASE2016 Proceedings

18. Description text Mining
Stack Traces Link to source code
Product/Component
Specific vocabulary
Suggestions ?
Data mining
inside !

19. Question Cases Precision Recall
Who should fix this
bug ?
Eclipse, Firefox, gcc
eclipse: 57%
firefox: 64%
gcc: 6%
—
How long will it take to
fix this bug ? (*)
JBoss
depends on the component
many similar reports: off by one hour
few similar reports: off by 7 hours
What is the severity of
this bug ? (**)
Mozilla, Eclipse, Gnome
mozilla,
eclipse:67% - 73%
gnome:
75%-82%
mozilla,
eclipse:50% - 75%
gnome:
68%-84%
Promising results but …
• how much training is needed ? (cross-project training ?)
• how reliable is the data ? (estimates, severity, assigned-to)
• does this generalise ? (on industrial scale ?)
replication is needed
(*) In CSMR2012 Proceedings
Who should fix this
bug ?
Eclipse, Firefox, gcc
eclipse: 57%
firefox: 64%
gcc: 6%
—
Irrelevant for
Practitioners
(**) In CSMR2011; MSR 2010 Proceedings

20. Story Points

21. M
achine
learning
inside
!
Human
MMRE: 0.48
Student Work
Learning Curve
In PROMISE2016 Proceedings

22. Emotion Analysis
“you’re one of the
worst offenders”
“scolding people [...] is not likely to encourage
people [...] to become senior developers”
“It’s been too
scary for years”
“I like offending
people, because I
think people who get
offended should be
offended”

23. Machine learning
inside !
Emotion Analysis — Results
Can human raters agree on the presence or
absence of emotions in issue reports?
In Empirical Software Engineering (Journal)

24. Q&A support

25. In CHI2016 Proceedings

26. version
control
build
developer
tests
deploy
scenario
tests
deploy to
production
measure &
validate

27. Agile Quality Assurance
Hype Cycle
27
Hype Cycle © Gartner
Visibility
Maturity
Technology
Trigger
Peak of
Inflated
Expectations
Trough of
Disillusionment
Slope of
Enlightenment
Plateau of
Productivity

28. Agile Quality Assurance
The Future ?
28
PersonalOpinion
Peak of
Inflated
Expectations
Hype Cycle © Gartner
Visibility
Maturity
Technology
Trigger
Trough of
Disillusionment
Slope of
Enlightenment
Plateau of
Productivity
IBM (Patents ) Eclipse
Microsoft Team Foundation Server
Google …
… for “in vivo” experiments