Farewell lecture given for the retirement of Professor Roel Wieringa, University of Twente, March 22nd 2019

1. Daniel Méndez
Technical University of Munich, Germany
www.mendezfe.org
Requirements Engineering Research
How good are we at solving practical problems?
@mendezfe
Farewell Lecture Roel Wieringa
Twente University, the Netherlands

2. 1968

3. 19771968

4. 19931977

5. 19971993

6. 19971993

7. 19971993

8. 19971993

9. How relevant is our research to industry?

10. Example: “Goal-oriented RE” (GORE)

11. Example: “Goal-oriented RE” (GORE)
Papers published [1]: 966
[1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016

12. Example: “Goal-oriented RE” (GORE)
Papers published [1]: 966
[1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016
Papers including a case study [1]: 131

13. Example: “Goal-oriented RE” (GORE)
Papers published [1]: 966
[2] Mavin, et al. Does Goal-Oriented Requirements Engineering Achieve its Goal?, 2017
[1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016
Papers including a case study [1]: 131
Studies involving practitioners [2]: 20

14. Example: “Goal-oriented RE” (GORE)
Papers published [1]: 966
[2] Mavin, et al. Does Goal-Oriented Requirements Engineering Achieve its Goal?, 2017
[3] Mendez et al. Naming the Pain in Requirements Engineering Initiative - www.re-survey.org
[1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016
Papers including a case study [1]: 131
Studies involving practitioners [2]: 20
Practitioners actually using GORE [3]: ~ 5%

15. Example: “Goal-oriented RE” (GORE)
Papers published [1]: 966
[2] Mavin, et al. Does Goal-Oriented Requirements Engineering Achieve its Goal?, 2017
[3] Mendez et al. Naming the Pain in Requirements Engineering Initiative - www.re-survey.org
[1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016
Papers including a case study [1]: 131
Studies involving practitioners [2]: 20
Practitioners actually using GORE [3]: ~ 5%
For comparison:
Icelanders believing in elves [4]:
[4] https://www.nationalgeographic.com/travel/destinations/europe/iceland/believes-elves-exist-mythology/
54%

16. Key question
How can we foster the
practical relevance of our research?

17. Key question
How can we foster the
practical relevance of our research?
This is a recognised problem

18. How can we foster the
practical relevance of our research?
What do practitioners think about

19. How can we foster the
practical relevance of our research?
Of course, we studied also that in more detail.
Franch Vogelsang Knauss Heldal Travassos CarverMéndez Coupette
What do practitioners think about

20. Study design (simplified)
Collection of 435 RE publications
from 2010-2016

21. Study design (simplified)
Collection of 435 RE publications
from 2010-2016
1-sentence summary for each
paper following same pattern
“A method to automatically recover
traceability links between various software
artefacts based on topic modelling”
“An experiment with students for
comparing two requirements elicitation
techniques when instantiating software
product lines (SPL) in order to understand
which approach is more suitable for eliciting
requirements in SPL”
A [contribution type] to [utility / scope] in
order to [improvement goal]

22. Study design (simplified)
Collection of 435 RE publications
from 2010-2016
1-sentence summary for each
paper following same pattern
Online survey with practitioners:
– assess the papers based on (sample) summaries
– propose research topics they find relevant
– demographics

23. Study design (simplified)
Collection of 435 RE publications
from 2010-2016
1-sentence summary for each
paper following same pattern
Online survey with practitioners:
– assess the papers based on (sample) summaries
– propose research topics they find relevant
– demographics
• 2164 ratings (positive and negative) and reasons
• 103 suggestions on RE research topics

24. Study design (simplified)
Collection of 435 RE publications
from 2010-2016
1-sentence summary for each
paper following same pattern
Online survey with practitioners:
– assess the papers based on (sample) summaries
– propose research topics they find relevant
– demographics
• 2164 ratings (positive and negative) and reasons
• 103 suggestions on RE research topics
Aggregation, anonymisation, analysis, disclosure (open data)

25. Why anonymisation is so important
“Fluffy topic”
“Who cares?”
“Not relevant
for my work”
Eckhardt, Vogelsang, Mendez.
Are “non-functional” requirements really non-
functional? An Investigation of Non-functional
Requirements in Practice. (ICSE 2016)
Racheva, Daneva, Sikkel Herrmann, Wieringa.
Do We Know Enough about Requirements Prioritization
in Agile Projects: Insights from a Case Study. (RE 2010)
“blablabla”
Rated unimportant
Rated worthwhile

26. How did practitioners rate the
relevance of the various research topics?

27. Rating by contribution topics
Essential
Worthwhile
Unwise
Unimportant
[Blended out for reasons of simplicity]
...
...

28. Rating by contribution topics
Essential
Worthwhile
Unwise
Unimportant
[Blended out for reasons of simplicity]
Everything seems to be
relatively interesting.
(To some, even goal
models seem essential.)
...
...

29. Rating by utility / scope facet
Essential
Worthwhile
Unwise
Unimportant

30. Rating by utility / scope facet
Essential
Worthwhile
Unwise
Unimportant
Even formal methods
are well perceived.

31. Rating by quality facet
Essential
Worthwhile
Unwise
Unimportant
usability
safety
reliability
performance
security
quality in general
functionality
sustainability
[Blended out for reasons of simplicity]
[Blended out for reasons of simplicity]

32. Rating by quality facet
Essential
Worthwhile
Unwise
Unimportant
usability
safety
reliability
performance
security
quality in general
functionality
sustainability
[Blended out for reasons of simplicity]
[Blended out for reasons of simplicity]
Every quality aspect
seems to matter. Safety,
Security, and Reliability
seem particularly
important.

33. If the topics are not the decisive factor,
what else could matter?

34. Does the conference venue matter?
Essential
Worthwhile
Unwise
Unimportant

35. Does the conference venue matter?
Essential
Worthwhile
Unwise
Unimportant
No.

36. Does the authors’ affiliation matter?
Essential
Worthwhile
Unwise
Unimportant

37. Does the authors’ affiliation matter?
Essential
Worthwhile
Unwise
Unimportant
No, but…
…the choice of track
seems to be indicative.

38. Does the research method matter?
Essential
Worthwhile
Unwise
Unimportant
Engineering Solution
Proposals
Observational Studies
Controlled Laboratory
Experiments
[Rest blended out
for reasons of
simplicity]
…
…

39. Does the research method matter?
Essential
Worthwhile
Unwise
Unimportant
Engineering Solution
Proposals
Observational Studies
Controlled Laboratory
Experiments
Yes!
Real-world environments
strongly preferred over
artificial (lab) environments.
[Rest blended out
for reasons of
simplicity]
…
…

40. Does the research method matter?
Essential
Worthwhile
Unwise
Unimportant
Engineering Solution
Proposals
Observational Studies
Controlled Laboratory
Experiments
Yes!
Real-world environments
strongly preferred over
artificial (lab) environments.
[Rest blended out
for reasons of
simplicity]
…
…

41. Key Take-Away
Management Aspects in Requirements Engineering
Agile Requirements Engineering
Human Factors in Requirements Engineering
….….
….….
‣ Every topic finds its champion in practice
‣ What matters most is how research is carried out:
We need to go out of the lab into the wild!

42. Key Take-Away
Management Aspects in Requirements Engineering
Agile Requirements Engineering
Human Factors in Requirements Engineering
….….
….….
‣ Every topic finds its champion in practice
‣ What matters most is how research is carried out:
We need to go out of the lab into the wild!
Told you so: “The only way to leave
idealised assumptions of the
laboratory is to enter the real world”
Wieringa and Morali. Technical Action Research as a Validation Method in Information
Systems Design Science, 2012.

43. Key Take-Away
Management Aspects in Requirements Engineering
Agile Requirements Engineering
Human Factors in Requirements Engineering
….….
….….
‣ Every topic finds its champion in practice
‣ What matters most is how research is carried out:
We need to go out of the lab into the wild!
Thank you!