This document outlines guidelines for empirical research methods in software engineering. It discusses case studies, experimental research, surveys, and post-mortem analysis. For each method, it provides examples and discusses how the method can be used to study software engineering problems. It also lists detailed guidelines for different aspects of empirical research, such as experimental context and design, data collection, analysis, and presentation and interpretation of results. The goal of the guidelines is to improve the quality and rigor of empirical studies in software engineering.
Empirical research methods for software engineering
1.
2.
3.
4.
5.
6.
7.
8. This overflow wasn’t monitored and therefore caused the entire control system to stop, and …boom!
9. The program causing the overflow wasn’t needed during flight, but only during initialization (up to -9s launch time).
10. It was kept running for 50 s into the flight to avoid re-initialization time of several hours in case of an aborted launch.
11. The software was designed for the Ariane 4 rocket, where this particular overflow could not happen.
12.
13. Case studies are also useful for describing and understanding rare events (such as disasters caused by software failures).
14.
15. Experimental are also referred to as research-in-small because their scope is limited.
16. When experimenting random tests are conducted.
17.
18.
19. Example: “If the president were elected tomorrow, whom would you vote?”
20. Surveys collect frequency data, but also information about reasons and preferences.
21. Example: “Why do you prefer a certain brand of car Surveys also test who holds certain preferences (male/female, age, ethnicity, income, location, etc
22. Surveys help understand why a certain phenomenon occurred and increase our ability to predict it.
23.
24.
25.
26.
27. The reviewer of a paper prior to its publication.
28. The authors of a paper.
29. Researchers planning an empirical study.
30.
31. Experimental design
32. Conduct of the experiment and Data collection
33. Analysis
34. Presentation of results
35.
36.
37.
38. Define the process by which the subjects and objects were selected.
39. Define the process by which subjects and objects are assigned to treatments.
40. Restrict yourself to simple study designs or, at least, to designs that are fully analyzed in the literature.