Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

SophiaConf 2018 - P. Urso (Activeeon)

98 vues

Publié le

Support de présentation : STAMP Test Amplification for DevOps

Publié dans : Technologie
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

SophiaConf 2018 - P. Urso (Activeeon)

  1. 1. 1 STAMP Test Amplification for DevOps Pascal Urso, PhD (Activeeon)
  2. 2. 2 Scenario – how to have good tests? Bugs detected by unit tests Bugs detected by IT Etc Etc Vision: test pyramid & bug filter How can I improve it? How can I reproduce it? How can I automatise it?
  3. 3. 3 Idea – Software Testing AMPlification l What about a software that produces test variants from your test cases? l What about a software that tests your tests? l What about a software that produces variants from your test configurations and execute your tests against them? l What about a software that reproduces a production bug simply analysing your production logs? l What about a test automation solution that let you emphasize test and quality in your DevOps processes?
  4. 4. 4 Idea – Software Testing AMPlification l A suite to increase your capacity to detect bugs - Descartes (“test your tests”) - DSpot (“amplify your tests”) - CAMP (Configuration AMPlification) - Evocrash (generate test case to reproduce bugs) l Use it by the means of standalone shell scripts l Available as Maven/Gradle plugins l Use it within Eclipse/IntelliJ IDE l Amplify your test practice
  5. 5. 5 Idea – Mutation l A (extreme) mutation is a (extreme) change in your code class A { int field = 3; public void Method(int inc) { field += 3; }} • • J.H. Andrews et al. 2006. Using mutation analysis for assessing and comparing testing coverage criteria. IEEE Trans on Software Engineering 32, 8 (2006), 608–624. • G. Petrovic et al. 2018. State of Mutation Testing at Google. In Proceedings of the International Conference on Software Engineering - Software Engineering in Practice (ICSE SEIP). Mutation class A { int field = 3; public void Method(int inc) { field -= 3; }} Extreme mutation class A { int field = 3; public void Method(int inc) { }}
  6. 6. 6 Idea – Descartes Idea – DSpot l Descartes l Fitness : does the test kills a mutant ? l I.e. does the test find bugs ? l Result : your test is “good” l (mutation score) l DSpot l Fitness : improved code coverage ? l Or improved mutation score! l Result : new tests ! Mutation Fitness evaluation stop ? mutants source code result tests Mutation Fitness evaluation stop ? mutants test result source code
  7. 7. 7 Idea – CAMP Configuration AMPlification
  8. 8. 8 Scenario – You need an infra
  9. 9. 9 9 • DSpot Wizard • Execute it and check results in Eclipse Console • Manage multiple configurations and executions How to – use DSpot within Eclipse
  10. 10. 10 10 • Create a freestyle job to run PIT Descartes • Add a post build action to generate STAMP reports • Execute it • Check trends How to – use Descartes within Jenkins
  11. 11. 11 11 • Execute your system test cases against multiple configurations • Use it as standalone or into your continuous delivery pipeline How to – execute CAMP
  12. 12. 12 12 • Feed Evocrash with your production logs • … Evocrash generates a test case able to reproduce the bug : just fix it • (saved 2 days to manually reproduce a crash : have a nice weekend :-)) How to – use Evocrash in Gradle/Maven
  13. 13. 13 How to – now you can have this
  14. 14. 14 14 How to – try it with STAMP virtual lab
  15. 15. 15 Conclusions l It’s not about “computers write software”: at least now, computers need people as much as we need them l Want to try it? Getting started for Dummies (see references) l Want to contribute? Github repo (see references) l Next steps: - Optimizations and enhancements - Microservices - More Jenkins, Github and Gitlab integrations - Metrics enhancements - More use cases and experiments - A lot of other stuff: stay tuned!
  16. 16. 16 References l STAMP Official site: https://www.stamp-project.eu/ l STAMP code: https://github.com/STAMP-project/ l STAMP for dummies: - https://github.com/STAMP-project/dspot/blob/master/docs/dspot-for-dummies.md - https://github.com/STAMP-project/pitest-descartes/blob/master/docs/descartes-for-dummies- mvn.md - More tutorials are coming…. l The Practical Test Pyramid - https://martinfowler.com/articles/practical-test-pyramid.html
  17. 17. 17 Thank you! Questions? Answers Tweets?