Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

DSpot - Test Amplification for Java

42 vues

Publié le

DSpot is a tool that Automatically detect and generate missing assertions for Junit test cases (also known as test amplification). For more information, see https://github.com/STAMP-project/dspot

Publié dans : Technologie
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

DSpot - Test Amplification for Java

  1. 1. Test amplification tutorial with DSpot Benjamin DANGLOT December 14th , 2017 benjamin.danglot@inria.fr STAMP Workshop Atos - Madrid 1/21
  2. 2. Use Case Example 2/21
  3. 3. DSpot
  4. 4. DSpot: Principle Program Test suite 3/21
  5. 5. DSpot: Principle × Program Test suite Test criterion 3/21
  6. 6. DSpot: Principle DSpot × Program Test suite Test criterion 3/21
  7. 7. DSpot: Principle DSpot × Program Test suite Test criterion Amplified test suite 3/21
  8. 8. DSpot: Principle DSpot Program Test suite Test criterion Amplified test suite 3/21
  9. 9. Use Case Example 4/21
  10. 10. Test Example 5/21
  11. 11. Test Example In blue, the input of the test 5/21
  12. 12. Test Example In blue, the input of the test In green, an oracle of the test to verify the current behavior 5/21
  13. 13. How DSpot works? 6/21
  14. 14. DSpot: 1. Input Modification 6/21
  15. 15. DSpot: 1. Input Modification Modifies the input to create new state of the program 7/21
  16. 16. DSpot: 1. Input Modification Removes a method call to create a new state of the program Remove method call 7/21
  17. 17. DSpot: 1. Input Modification Removes a method call to create a new state of the program 7/21
  18. 18. DSpot: 1. Input Modification Removes a method call to create a new state of the program Before: benjamin.isHungry() → false After: benjamin.isHungry() → true 7/21
  19. 19. DSpot: 2. Assertion Generation 8/21
  20. 20. DSpot: 2. Assertion Generation Goal: Generate assertions to verify the state of the program 1. Remove existing assertions 2. Instrument the test 3. Run Instrumented test to collect values 4. Generate assertions based on values collected 9/21
  21. 21. DSpot: 2. Assertion Generation A. Removes existing assertions 10/21
  22. 22. DSpot: 2. Assertion Generation assertion removed A. Removes existing assertions 11/21
  23. 23. DSpot: 2. Assertion Generation B. Instruments the code to gather new values 12/21
  24. 24. DSpot: 2. Assertion Generation B. Instruments the code to gather new values 13/21
  25. 25. DSpot: 2. Assertion Generation C. Runs the instrumented test and obtains observations 14/21
  26. 26. DSpot: 2. Assertion Generation C. Runs the instrumented test and obtains observations Observations: benjamin.isHungry() → true benjamin.isHappy() → false 15/21
  27. 27. DSpot: 2. Assertion Generation D. Generates new assertions based on observations Observations: benjamin.isHungry() → true benjamin.isHappy() → false 16/21
  28. 28. DSpot: 2. Assertion Generation D. Generates new assertions based on observations Observations: benjamin.isHungry() → true benjamin.isHappy() → false 17/21
  29. 29. DSpot: Amplification of Test Results Original test Amplified test 18/21
  30. 30. DSpot: Amplification of Test Results Original test Method call removed Assertion Generated Amplified test 18/21
  31. 31. DSpot: 3. Test Selection 19/21
  32. 32. Pre-Requisite: Mutation Analysis Test Suite Program The mutation analysis computes the mutation score, a measure of the efficiency of tests 20/21
  33. 33. Mutation Analysis Test Suite Program Variant 1 Test Suite Program Variant 1 Inject Fault First, creates variants of program by injecting faults 20/21
  34. 34. Mutation Analysis Test Suite Program Variant 1 Variant 2 Test Suite Program Variant 1 Variant 2Inject Fault Inject Fault First, creates variants of program by injecting faults 20/21
  35. 35. Mutation Analysis Test Suite Program Variant 1 Variant 2 Variant 3 Test Suite Program Variant 1 Variant 2 Variant 3 Inject Fault Inject Fault Inject Fault First, creates variants of program by injecting faults 20/21
  36. 36. Mutation Analysis Test Suite Program Variant 1 Variant 2 Variant 3Run Run Run Then, runs tests against each variant 20/21
  37. 37. Mutation Analysis Test Suite Program Variant 1 Variant 2 Variant 3Run Run Run Killed Killed Alive Failing test ⇔ mutant killed ⇔ behavior variation detected 20/21
  38. 38. Selection Of Test Using Mutation Analysis 21/21
  39. 39. Selection Of Test Using Mutation Analysis 21/21
  40. 40. Selection Of Test Using Mutation Analysis 21/21
  41. 41. Selection Of Test Using Mutation Analysis 21/21
  42. 42. Selection Of Test Using Mutation Analysis 21/21
  43. 43. Selection Of Test Using Mutation Analysis 21/21
  44. 44. Selection Of Test Using Mutation Analysis 21/21
  45. 45. Selection Of Test Using Mutation Analysis 21/21
  46. 46. DSpot: iteration 22/21
  47. 47. DSpot: Try it https://github.com/danglotb/tavern 23/21
  48. 48. DSpot: Command Line Options https://github.com/danglotb/tavern • --path-to-properties src/main/resources/tavern.properties: general description of the project • --iteration 2: number of time that the main loop is applied • --test fr.inria.stamp.MainTest: full qualified name of the test class to be amplified • --cases test: name of the test method (test case) • --amplifiers TestDataMutator: used input modification • --test-criterion PitMutantScoreSelector: test criterion selector used • --verbose: enable more logs • --path-pit-result mutations results/original.csv: read mutation analysis from file to speed up the exe 24/21
  49. 49. Input Modification: Available Operators • Literal Modification 1. Add/Remove/Replace a char in String 2. replace by Random String 3. +1/−1/×2/÷2 values 4. replace by random/upper bound/lower bound value 5. negate boolean • Method Call Duplication • Method Call Deletion • Generation of Method Call (and required objects) 25/21
  50. 50. Test Selection: Available Selector • Mutation Score • Branch Coverage • Instruction Coverage and Execution Path • Behavior Changes 26/21
  51. 51. DSpot: Playing With Command Line Options https://github.com/danglotb/tavern Input Modification --amplifiers: • MethodAdd: duplicate method call • MethodRemove: remove method call • TestDataMutator: mutate literals • StatementAdd: add method call Compose them using ’:’ Test Criterion Selector --test-criterion: • PitMutantScoreSelector: increase mutation score • JacocoCoverageSelector: increase coverage/exec. path • TakeAllSelector: take all amplified test You can also play with --iteration 27/21

×