Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Modern Code Reviews in Open-Source Projects: Which Problems Do They Fix?

112 vues

Publié le

Modern Code Review in a Feedback-Driven Development World with Continuous Integration on Travis CI

Publié dans : Logiciels
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Modern Code Reviews in Open-Source Projects: Which Problems Do They Fix?

  1. 1. Modern Code Reviews in Open-Source Projects: Which Problems Do They Fix? Moritz Beller Delft University of Technology @Inventitech Andy Zaidman, Georgios Gousios, Alberto Bacchelli, Elmar Jürgens, Radjino Bohlanath, Shane McIntosh
  2. 2. Modern Code Review Workflow
  3. 3. Modern Code Review Workflow Old
  4. 4. Modern Code Review Workflow
  5. 5. Modern Code Review Workflow
  6. 6. Modern Code Review Workflow
  7. 7. Modern Code Review Workflow
  8. 8. Modern Code Review Workflow
  9. 9. Modern Code Review Workflow
  10. 10. Modern Code Review Workflow
  11. 11. Modern Code Review Workflow
  12. 12. Modern Code Review Workflow
  13. 13. Modern Code Review Workflow
  14. 14. Modern Code Review Workflow +
  15. 15. /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread();
  16. 16. /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread();
  17. 17. /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread();
  18. 18. /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ public Runnable watchDogUiThread = new StartupUIThread();
  19. 19. /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ public Runnable watchDogUiThread = new StartupUIThread();
  20. 20. Review-triggered Change /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ public Runnable watchDogUiThread = new StartupUIThread();
  21. 21. /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread();
  22. 22. /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread();
  23. 23. /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ private Runnable watchDogUiThread = new StartupUIThread();
  24. 24. /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ private Runnable watchDogUiThread = new StartupUIThread();
  25. 25. Self-motivated + Review-triggered Change /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ private Runnable watchDogUiThread = new StartupUIThread();
  26. 26. /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread();
  27. 27. /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread();
  28. 28. /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread(); /** * The ui thread for WatchDog registration. From * this thread, the WatchDog program is started. */ public Runnable watchDog = new StartupUIThread();
  29. 29. /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread(); /** * The ui thread for WatchDog registration. From * this thread, the WatchDog program is started. */ public Runnable watchDog = new StartupUIThread();
  30. 30. Self-motivated Change /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread(); /** * The ui thread for WatchDog registration. From * this thread, the WatchDog program is started. */ public Runnable watchDog = new StartupUIThread();
  31. 31. /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread();
  32. 32. /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread();
  33. 33. /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread();
  34. 34. No Change /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread();
  35. 35. No Change /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread();
  36. 36. No Change /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ // TODO (GG) Provide a more descriptive name public Runnable watchDog = new StartupUIThread(); /** The ui thread for WatchDog registration. */ public Runnable watchDog = new StartupUIThread();
  37. 37. Does it matter?
  38. 38. >1,400 changes >245 tasks
  39. 39. Motivation for Change
  40. 40. Motivation for Change
  41. 41. 78-90% Motivation for Change
  42. 42. 78-90% Motivation for Change 22-10%
  43. 43. 78-90% Motivation for Change 22-10%
  44. 44. Motivation for No Change
  45. 45. Motivation for No Change
  46. 46. Motivation for No Change 7-35%
  47. 47. Motivation for No Change 7-35%
  48. 48. Type of Change
  49. 49. Doc Visual Structure Resource Check Interface Logic Larger Support ConQAT Gromacs 10% 20% 30% 40%
  50. 50. 75 Doc Visual Structure Resource Check Interface Logic Larger Support ConQAT Gromacs 10% 20% 30% 40%
  51. 51. 75 : Doc Visual Structure Resource Check Interface Logic Larger Support ConQAT Gromacs 10% 20% 30% 40%
  52. 52. 75 : 25 Doc Visual Structure Resource Check Interface Logic Larger Support ConQAT Gromacs 10% 20% 30% 40%
  53. 53. 75 : 25
  54. 54. What Influences Code Review?
  55. 55. What Influences Code Review?
  56. 56. What Influences Code Review?
  57. 57. What Influences Code Review? > 2,800 changes > 970 tasks
  58. 58. What Influences Code Review? > 2,800 changes > 970 tasks
  59. 59. What Influences Code Review? > 2,800 changes > 970 tasks
  60. 60. What Influences Code Review? > 2,800 changes > 970 tasks
  61. 61. What Influences Code Review? > 2,800 changes > 970 tasks
  62. 62. What Influences Code Review? > 2,800 changes > 970 tasks
  63. 63. What Influences Code Review? > 2,800 changes > 970 tasks
  64. 64. What Influences Code Review? > 2,800 changes > 970 tasks
  65. 65. “Continuous” Integration Then (2014) +
  66. 66. “Continuous” Integration Then (2014) +
  67. 67. “Continuous” Integration Then (2014) +
  68. 68. “Continuous” Integration Then (2014) +
  69. 69. Continuous Integration Now (2017)
  70. 70. Continuous Integration Now (2017) CI Services
  71. 71. Continuous Integration Now (2017) a) Static Analysis CI Services
  72. 72. Continuous Integration Now (2017) a) Static Analysis b) Dynamic Analysis CI Services
  73. 73. Continuous Integration Now (2017) + a) Static Analysis b) Dynamic Analysis CI Services
  74. 74. a) How Prevalent Is Static Analysis?
  75. 75. 122 a) How Prevalent Is Static Analysis?
  76. 76. 122 a) How Prevalent Is Static Analysis?
  77. 77. 122 a) How Prevalent Is Static Analysis?
  78. 78. 122 122 a) How Prevalent Is Static Analysis?
  79. 79. 122 122 a) How Prevalent Is Static Analysis?
  80. 80. 122 36 122 a) How Prevalent Is Static Analysis?
  81. 81. 122 36 122 a) How Prevalent Is Static Analysis?
  82. 82. 122 36 122 a) How Prevalent Is Static Analysis?
  83. 83. a) How Prevalent Is Static Analysis?
  84. 84. Source Amount of Projects Using ASATs Using >1 ASAT Enforcing ASAT 122 59% 23% - 36 77% 36% 36% a) How Prevalent Is Static Analysis?
  85. 85. Source Amount of Projects Using ASATs Using >1 ASAT Enforcing ASAT 122 59% 23% - 36 77% 36% 36% a) How Prevalent Is Static Analysis?
  86. 86. Source Amount of Projects Using ASATs Using >1 ASAT Enforcing ASAT 122 59% 23% - 36 77% 36% 36% a) How Prevalent Is Static Analysis?
  87. 87. Source Amount of Projects Using ASATs Using >1 ASAT Enforcing ASAT 122 59% 23% - 36 77% 36% 36% a) How Prevalent Is Static Analysis?
  88. 88. Source Amount of Projects Using ASATs Using >1 ASAT Enforcing ASAT 122 59% 23% - 36 77% 36% 36% a) How Prevalent Is Static Analysis?
  89. 89. Source Amount of Projects Using ASATs Using >1 ASAT Enforcing ASAT 122 59% 23% - 36 77% 36% 36% Moritz Beller, Radjino Bholanath, Shane McIntosh, Andy Zaidman: Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Source Software, SANER, Osaka (Japan), 2016 a) How Prevalent Is Static Analysis?
  90. 90. TODO: Add background with Sun b) How Prevalent Is Dynamic Analysis?
  91. 91. TODO: Add background with Sun CI is the best practice in which developers not only integrate their work into a shared mainline frequently, but also verify the quality continuously through testing. b) How Prevalent Is Dynamic Analysis?
  92. 92. TODO: Add background with Sun CI is the best practice in which developers not only integrate their work into a shared mainline frequently, but also verify the quality continuously through testing. It is known. b) How Prevalent Is Dynamic Analysis?
  93. 93. <TL;DR>...</TL;DR>
  94. 94. <TL;DR>...</TL;DR> ------------------------------------------------------- T E S T S ------------------------------------------------------- Running nl.tudelft.watchdog.ClientVersionCheckerTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec Results : Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
  95. 95. <TL;DR>...</TL;DR> ------------------------------------------------------- T E S T S ------------------------------------------------------- Running nl.tudelft.watchdog.ClientVersionCheckerTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec Results : Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 <TL;DR>...</TL;DR>
  96. 96. <TL;DR>...</TL;DR> ------------------------------------------------------- T E S T S ------------------------------------------------------- Running nl.tudelft.watchdog.ClientVersionCheckerTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec Results : Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 <TL;DR>...</TL;DR>
  97. 97. <TL;DR>...</TL;DR> ------------------------------------------------------- T E S T S ------------------------------------------------------- Running nl.tudelft.watchdog.ClientVersionCheckerTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec Results : Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 <TL;DR>...</TL;DR>
  98. 98. <TL;DR>...</TL;DR> ------------------------------------------------------- T E S T S ------------------------------------------------------- Running nl.tudelft.watchdog.ClientVersionCheckerTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec Results : Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 <TL;DR>...</TL;DR>
  99. 99. <TL;DR>...</TL;DR> ------------------------------------------------------- T E S T S ------------------------------------------------------- Running nl.tudelft.watchdog.ClientVersionCheckerTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec Results : Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 <TL;DR>...</TL;DR> ● Parsing Log files for Java: Maven, Gradle, Ant
  100. 100. <TL;DR>...</TL;DR> ------------------------------------------------------- T E S T S ------------------------------------------------------- Running nl.tudelft.watchdog.ClientVersionCheckerTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec Results : Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 <TL;DR>...</TL;DR> ● Parsing Log files for Java: Maven, Gradle, Ant
  101. 101. <TL;DR>...</TL;DR> ------------------------------------------------------- T E S T S ------------------------------------------------------- Running nl.tudelft.watchdog.ClientVersionCheckerTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec Results : Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 <TL;DR>...</TL;DR> ● Parsing Log files for Java: Maven, Gradle, Ant
  102. 102. <TL;DR>...</TL;DR> ------------------------------------------------------- T E S T S ------------------------------------------------------- Running nl.tudelft.watchdog.ClientVersionCheckerTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec Results : Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 <TL;DR>...</TL;DR> ● Parsing Log files for Java: Maven, Gradle, Ant ● Parsing Log files for Ruby: bundler and rake
  103. 103. <TL;DR>...</TL;DR> ------------------------------------------------------- T E S T S ------------------------------------------------------- Running nl.tudelft.watchdog.ClientVersionCheckerTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec Results : Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 <TL;DR>...</TL;DR> ● Parsing Log files for Java: Maven, Gradle, Ant ● Parsing Log files for Ruby: bundler and rake
  104. 104. b) How Prevalent Is Dynamic Analysis?
  105. 105. ● 69% of Java and 87.5% of the Ruby projects: test runs b) How Prevalent Is Dynamic Analysis?
  106. 106. ● 69% of Java and 87.5% of the Ruby projects: test runs ● Overall, 81% of the projects: test runs b) How Prevalent Is Dynamic Analysis?
  107. 107. ● 69% of Java and 87.5% of the Ruby projects: test runs ● Overall, 81% of the projects: test runs ● 96% of builds: test runs b) How Prevalent Is Dynamic Analysis?
  108. 108. b) How Prevalent Is Dynamic Analysis?
  109. 109. b) How Prevalent Is Dynamic Analysis?
  110. 110. b) How Prevalent Is Dynamic Analysis?
  111. 111. Testing is the single most important reason for broken builds, more prevalent than compile errors, missing dependencies, build cancellations and provisioning problems together. b) Testing Is Central To CI
  112. 112. Testing is the single most important reason for broken builds, more prevalent than compile errors, missing dependencies, build cancellations and provisioning problems together. b) Testing Is Central To CI Moritz Beller, Georgios Gousios, Andy Zaidman: Oops, My Tests Broke the Build: An Explorative Analysis of Travis CI with GitHub, MSR, Buenos Aires (Argentina), 2017
  113. 113. What Is TravisTorrent?
  114. 114. What Is TravisTorrent? MSR Mining Challenge 2017 Public Data Set
  115. 115. What Is TravisTorrent? Free* SQL-like cloud query interface on Google BQ MSR Mining Challenge 2017 Public Data Set
  116. 116. What Is TravisTorrent? .testroots.org Free* SQL-like cloud query interface on Google BQ MSR Mining Challenge 2017 Public Data Set
  117. 117. TODO: Add background with Sun Research on Continuous Code Review Enabled By TravisTorrent
  118. 118. TODO: Add background with Sun - git_branch - git_all_built_commits - ... Research on Continuous Code Review Enabled By TravisTorrent
  119. 119. TODO: Add background with Sun - git_branch - git_all_built_commits - ... - gh_project_name - gh_is_pr - gh_team_size - gh_diff_tests_added - gh_first_commit_... Research on Continuous Code Review Enabled By TravisTorrent
  120. 120. TODO: Add background with Sun - git_branch - git_all_built_commits - ... - gh_project_name - gh_is_pr - gh_team_size - gh_diff_tests_added - gh_first_commit_... - tr_prev_build - tr_status - tr_log_tests_ran - tr_log_tests_failed - ... Research on Continuous Code Review Enabled By TravisTorrent
  121. 121. TODO: Add background with Sun - git_branch - git_all_built_commits - ... - gh_project_name - gh_is_pr - gh_team_size - gh_diff_tests_added - gh_first_commit_... - tr_prev_build - tr_status - tr_log_tests_ran - tr_log_tests_failed - ... 62columns Research on Continuous Code Review Enabled By TravisTorrent
  122. 122. TODO: Add background with Sun - git_branch - git_all_built_commits - ... - gh_project_name - gh_is_pr - gh_team_size - gh_diff_tests_added - gh_first_commit_... - tr_prev_build - tr_status - tr_log_tests_ran - tr_log_tests_failed - ... 62columns Research on Continuous Code Review Enabled By TravisTorrent
  123. 123. TODO: Add background with Sun - git_branch - git_all_built_commits - ... - gh_project_name - gh_is_pr - gh_team_size - gh_diff_tests_added - gh_first_commit_... - tr_prev_build - tr_status - tr_log_tests_ran - tr_log_tests_failed - ... 62columns Research on Continuous Code Review Enabled By TravisTorrent
  124. 124. @Inventitech Moritz Beller, TU Delft Modern Code Reviews in Open-Source Projects: Which Problems Do They Fix?
  125. 125. @Inventitech Moritz Beller, TU Delft Modern Code Reviews in Open-Source Projects: Which Problems Do They Fix?
  126. 126. @Inventitech Moritz Beller, TU Delft Modern Code Reviews in Open-Source Projects: Which Problems Do They Fix?
  127. 127. @Inventitech Moritz Beller, TU Delft Modern Code Reviews in Open-Source Projects: Which Problems Do They Fix?

×