ICSE 2022 keynote

1. Mathematicians, Social Scientists,
or Engineers?
The Split Minds of Software
Engineering Researchers
Lionel Briand
http://www.lbriand.info
ICSE 2022

2. A Word of Thanks …
PhD advisor: Victor Basili
Collaborators and students: See the 2020 ACM SE Notes
interview on my web page
Family, for their patience and tolerance ;-)
2

3. Motivations
Happiness is not a function of what you achieve. It's a function
of how you spend your time. Success is a temporary thrill.
Happiness lies in doing daily activities that bring you joy.
There's always a new mountain to climb. You don't have to
anchor your emotions to the summit.
Adam Grant, Organizational Psychologist, Wharton, USA
3

4. What Kept Me in Academia
4

5. Who are we?
What is our purpose?
5

6. Traditional Engineering Fields
6

7. Software Systems
• No standard education
• No standard practices
• Human intensive
7
• Variability
• Frequent change
• No continuity property

8. A Tale of Three Communities
8
Formal Methods
and
Guarantees
Human & Social
Studies
Engineering
Automated
Solutions

9. A Tale of Three Communities
9
Formal Methods
and
Guarantees
Human & Social
Studies
Engineering
Automated
Solutions
• Combining solvers and
smart heuristics
• Probabilistic guarantees
• Trade-offs
• Scalability
• Applicability

10. A Tale of Three Communities
10
Formal Methods
and
Guarantees
Human & Social
Studies
Engineering
Automated
Solutions
In Vitro vs.
in Vivo
evaluation
• Assumptions
• Complexity
• Limitations

11. Reality
11
Formal Methods
and
Guarantees
Human & Social
Studies
Engineering
Automated
Solutions
Disconnect
Twitter
Stack Overflow
Diversity
Values
Leadership
Coordination
Sleep
Cannabis
…
Evolutionary computing
Machine learning
Natural Language Proc.
…
Solvers
Theorem provers
Symbolic execution
…

12. Representation
12
Formal Methods
and
Guarantees
Human & Social
Studies
Engineering
Automated
Solutions
Distinguished
papers ICSE 2022
(nominated)
7
10
2
Original
emphasis

13. A World Apart
• “HiTyper iteratively conducts static inference and DL-based
prediction until the TDG is fully inferred. Experiments on two
benchmark datasets show that HiTyper outperforms state-of-the-
art DL models …”
• “Leadership: (1) is dynamically shared among team members; (2)
engenders a sense of belonging to the team; and (3) involves
balancing competing organizational cultures …”
• “Diva proves 68% more theorems than TacTok and 77% more than
ASTactic. Complementary to CoqHammer, Diva proves 781
theorems (27% added value) that CoqHammer does not, and 364
theorems no existing tool has proved automatically.”
13

14. A World Apart
• “HiTyper iteratively conducts static inference and DL-based
prediction until the TDG is fully inferred. Experiments on two
benchmark datasets show that HiTyper outperforms state-of-the-
art DL models …”
• “Leadership: (1) is dynamically shared among team members; (2)
engenders a sense of belonging to the team; and (3) involves
balancing competing organizational cultures …”
• “Diva proves 68% more theorems than TacTok and 77% more than
ASTactic. Complementary to CoqHammer, Diva proves 781
theorems (27% added value) that CoqHammer does not, and 364
theorems no existing tool has proved automatically.”
14
Different backgrounds, language,
research methods …

15. Diversity is Good
• This is what a field like software engineering requires
• But …
• Those communities need to be (significantly) connected
• We need to be able to talk to each other and work together
• Only then we will be able to offer the novel solutions that
software engineering needs and maximize our impact
15

16. Would it change anything for
the three sub-communities if
we attended different
conferences and published
in different journals?
16

17. Social Studies and Football
• Regarding human and social science studies …
• Assuming you replace “software engineering” by “football” in
a paper …
• If the paper abstract still makes sense, is it software
engineering research?
• Does the software engineering context matter?
17

18. Biases in Engineering Research
• What is the impact of our heavy reliance on Open-Source systems
(OSS) and data for our engineering research?
• Type and size of systems, underlying technology
• Generalization of results?
• Context is everything is software engineering …
18

19. Formal Methods for Software
Engineering
• Software engineering research versus applied mathematics,
Computer Aided Verification
• Can it be so practically limited, so disconnected from
engineering reality, that it can only be seen as an intellectual
exercise?
• Is such research a necessary first step?
19

20. Impact
21

21. Dimensions of Impact
• Timeline: Short versus long term
• Mechanism: Publications, interactions, knowledge transfer,
organizational change, commercialization, …
• Evidence: Empirical results, industrial adoption, …
22

22. My Experience with Industry
• Scope the collaboration
• Define the problem(s) and prioritizing
• Read the research literature
• Identify the gaps
• Plan the research
• ….
23

23. Typical (abstract) Dialog
• Partner: What are the solutions available in the research literature?
• Me (Slightly embarrassed): Well, there are good ideas but nothing
that can fit your needs and scale to your systems. We have a great
deal to do, and it is going to take X years.
• Lack of operational, scalable, practical solutions
• Recent example: Regression testing, a topic that has been,
relatively speaking, extensively investigated
• No accurate, applicable and scalable (published) solutions in a CI
context, with frequent builds and large systems
24

24. Schneiderman’s Model
• Ben Schneiderman, “The New ABCs of Research: Achieving Breakthrough Collaborations”, 2016
• Linear model of research is ineffective
• A Collaborative Model of Research and Innovation
• Access to smart and motivated professionals
• Produce dual benefits: publishable research results and focused practical solutions that serve an
existing or foreseeable market
25
Basic Research
Applied
Research
Innovation & Development

25. Schneiderman’s Model
• Applied research feeds basic research
• Lower barriers to technology transfer
• Common practice in GAFAM
• Public-private partnerships
26
Basic Research
Applied
Research
Innovation & Development

26. Strategy for Software
Engineering
27
• Research driven by industry needs
• Realistic evaluations
• Combining research with technology transfer and innovation
Problem
Formulation
Problem
Identification
State of the
Art Review
Candidate
Solution(s)
Initial
Validation
Training
Realistic
Validation
Industry Partners
Research Groups
1
2
3
4
5
7
Solution
Release
8
6
Adapted from Gorschek et al., 2006

27. Making it Possible
• Institutions: Support partnerships (e.g., technology transfer
offices, enabling industrial leaves for faculty members)
• Funding Agencies: Finance partnerships through specific
programs
• Private companies: Long term commitments, open science
• Academics: Develop an appreciation for demand-driven
research in industrial contexts. Change publication model.
28

28. The goal is not to forbid
curiosity-driven research
but to encourage demand-
driven, collaborative
research
29

29. Mathematicians, Social Scientists,
or Engineers?
The Split Minds of Software
Engineering Researchers
Lionel Briand
http://www.lbriand.info
ICSE 2022